Sound (not noise) reduction, a headscratcher

[tulamide]

I'll just leave it here, for brainstorming, tips, solutions or helpful comments:

I'm a fan of Hermitcraft. There, I said it. Hermitcraft is a bunch of Youtubers, called Hermits, playing together on a closed Minecraft Server. So far so boring for you all. But here comes the interesting part. I recently joined the Hermitcraft remixers. Fans that know a bit about music, who rip Hermits talking or singing and make a song out of it.

Here are two examples.
Elybeatmaker - Mumbo AFK
tulamide - Poultry Man

Now with that came issues that I never had to deal with before. Unwanted sounds. Have a listen to this example (first 30 seconds or so), you will notice background music and the opening of a barrel while they speak. https://youtu.be/2e3OqCwMQXo?t=1248

How would one go about reducing or even eliminating those unwanted sounds? I tried hard, but a lot of the frequencies are in the band of the voices. A gate does also not really work, since it opens on the voices. A fingerprint noise reduction test also failed, probably because it is not a static sound.

Of course, an AI would maybe notice what belongs to where and could remove the unwanted bits, but I have no AI, nor would I know how to program one.

But you guys are masters of frequencies and digital signal processing. If you don't know a way, who else would? And so I ask right here on the forum. What do you think, what could be done?

[martinvicanek]

Difficult. The good thing is that the background music is at a low level and is perceived only in a certain band which, however, overlaps with the signal that you want to keep. The best you can do is try to get a fingerprint of the background noise (which is not easy because the guys talk all the time) and then try a noise removal tool. I added fingerprint capability to viewtopic.php?f=3&t=12863&start=0 some time ago but never posted it, maybe I will.

At any rate, the result will never be entirely unaffected by the surgery.

[tulamide]

Difficult. The good thing is that the background music is at a low level and is perceived only in a certain band which, however, overlaps with the signal that you want to keep. The best you can do is try to get a fingerprint of the background noise (which is not easy because the guys talk all the time) and then try a noise removal tool. I added fingerprint capability to viewtopic.php?f=3&t=12863&start=0 some time ago but never posted it, maybe I will.

At any rate, the result will never be entirely unaffected by the surgery.

I expect it being affected, and I'm looking for a good balance between sound removal and original voice character.

I thought a lot about fingerprint noise removal (as I said the test went bad, but maybe it was a bad fingerprint). I came to an idea that won't work with standard noise removal tools, but maybe with something build by this lovely community! I can say for sure it would be worth it, because every HC Remixer would be eternally thankful for such a tool.

My idea is based on the fact, that
a) all sounds of Minecraft are accessible (I think they used to be ogg, but now are embedded in json, however, I could get them and provide them as wav)
b) all Remixers know the sounds very well, and so can select the correct sound file when they hear a sound.

Forget about the background music, that's not so difficult. But, the sounds that are played, when the players interact with the game world, they should be reduced or removed. Is there any chance that a plugin, having the original sound at hand, could go through the vocal track and reduce or remove all occurances of that sound?

These properties are very likely: No Youtuber changes the pitch of the sounds, but all Youtubers of course have to compress their video to Youtube format (that should be mp3 128 bit, right?). That's why I think the tool would have to be tolerant to a tiny degree of difference.

If the majority thinks that something like that is impossible to create, I'll accept my defeat