Okay...I'm going to give a small 101 on IP law. Gonna say it right now: technically, all fan projects of games are indeed illegal under IP laws. You might be saying "then why don't they get shut down all the time?" Well...there's a few reasons for this (1/?).

1. The fan project is done and is circulating on the Internet and there's no way to quell that circulation because Internet.
2. There are a ton of fan projects in the works for all sorts of games out there, from mods to hacks.
3. Most companies just turn a blind eye to it. (2/?)

In the case of #3, can they shut it down? Yes? Do they? Most of the time, no. Often time a company won't if they know their reputation would be damaged doing so and/or there is no financial gain for the project creator. (3/?)

So of course, if you attempt to make money off of a fan project, you will very likely get shut down. This includes providing prizes in a competition that involved modded versions of games. (4/?)

So now today's point of contention that brought this all up: Big House. If you don't know, Big House is (pardon the pun) one of the bigger Smash Bros. competitions in the US, but its online competition was just Cease and Desisted by Nintendo. (5/?)

Because of that aforementioned "prize" statement. Big House was going to run Melee...online. There is currently a mod out there that allows rollback netcode to play Melee online. Of course, Nintendo could have C&D'd the mod to begin with but they (shockingly) didn't. (6/?)

They were in their legal right to do so. Remember: under IP law, all fan projects are illegal but they took option #3 (see above) and just turned a blind eye to it. Big House, however, monetized that mod because of the entry fee to play and the prize pool. (7/?)

As a result, of course Nintendo was going to stop it. That is taking an established IP, under IP law, and monetizing it, which is way over the line for a lot of companies with their IPs. (8/?)

You might also bring this up: what about that time they tried to stop a Melee tournament in the mid-2010s? They backed off since charity money was put to allow Melee to be featured on the big stage and Nintendo's reputation getting shot in the foot wasn't worth stopping it. (9/?)

And also you might bring up stuff like AM2R or Mother 3. For the former, honestly, that was most likely just bad timing on when AM2R was making the rounds because Samus Returns was in development. With both AM2R and Mother 3, they could C&D all they want but...(10/?)

...they were already circulating online and there was really no way to stop it so it's Option #1. (11/?)

Look, I get it. Copyright and IP law is really in the grey area and really needs to be updated to clarify things but it exists to prevent situations like this from going too far out of the IP holder's control. (12/?)

Believe me, I absolutely hate that Nintendo shuts down fan projects left and right, especially when almost all of them are harmless. I get that they are in their legal right to shut them down, but that doesn't make them morally right to do so. (13/?)

But in the main example in this thread (Big House), they were legally and morally right here. In addition, they are not trying to shut Melee down. If they were...why didn't they stop Slippi prior to it being circulated? (14/?)

The issue here is that there was money involved with a modded (i.e. fan) version of a Nintendo IP, so of course they are going to get involved. Heck, Big House was warned by other TOs this would happen and they did it anyway. (15/?)

The moral of all of this is: if you want to make a fan project, tread with caution because the IP holder can legally shut it down at anytime and do NOT, NEVER, try to have money attached to the fan project. In any way. Thank you for reading. ^-^ (16/16)

Addendum: I should note this is copyright law, but IPs are copyrighted so...yeah. Anyway, here's a site you can read up on with that for US copyright law. This link specifically leads to what copyright infringement is.
https://www.copyright.gov/title17/92chap5.html