I understand that, and understand why you’d want that (it’s a Utopian ideal, after all).
The issue is that while ending copyright completely would make a lot of things better, and make a lot of works more viable/legal (specifically derivatives of existing works), it would also DESTROY a whole lot of viability of works, and make the vast majority completely unfeasible.
The simple fact is that with no copyright, it would be almost impossible for authors to make a living. While some musicians might be able to live off of live performances, that leaves non-performing musicians with no way to make a living. Writers do not typically perform their works live, so they’re left destitute as well.
Another form of art that would be almost completely destroyed without any copyright would be TV shows and films. The amount of money required for such endeavors are large, and without copyright there would be absolutely no reasonable way for them to make their money back - and without that, they’d never get made in the first place.
The (stated) purpose of copyright is to encourage authorship in science and the arts by providing a legal means by which authors can get paid for their works. It’s a necessary legal concept to ensure not just that the public pays (which some object to), but also to ensure that corporations don’t simply take whatever they want from authors without paying them.
The issue is that copyright was always intended as a balancing act between the encouragement of authorship and expanding public art and knowledge. The balancing factor on one side is that you want to give as much as you need to authors to make them create. The balancing factor on the other is that you want to make the timeframe and restrictions as little as possible to expand the public domain. The current system gives far too much power to authors (or, more specifically, to “owners” as these days they’re often not the authors) and far too little back to the public (currently, nothing). Your solution (end all copyright) swings the pendulum fully the other way - the public gets everything, but the author gets nothing. This “solution” is just as bad as the current situation, if not worse. Fixing copyright lies in finding the proper balance between the two.