> We could train a model on a dataset where any reference to consciousness is thoroughly removed. Then explore whether the model can bootstrap a discussion about consciousness.

I'm less interested in the philosophy than the practical "how directly does training data influence model behavior" question, however PleIAs' SYNTH dataset [1] used to train the Baguettotron model has a small set of "self-awareness about training condition" documents which are amplified to (attempt to) give the expected answers to "what are you?" questions, filtering out any rows with "Pleias self-knowledge" under "query_seed_url" might be a good first stab at such a dataset

[1] https://huggingface.co/datasets/PleIAs/SYNTH