There is an option in Settings to use the music history played on this device (whether phone, iPad, or AppleTV) which sounds like the kind of toggle you are looking for. The setting is on by default but you can turn it off temporarily when playing something you don’t want influencing your algorithm.
Apple Vision Pro headset. It’s way too expensive but the quality is so damn amazing. I’m a VR developer so I appreciate its power.