While we should not confuse our experience as the totality of reality, neither should we make unverifiable assumptions that a totality of reality exists outside of our experiences.
Ten people stare at a blank surface
Half of them are wearing blue-tinted glasses.
Half of them are wearing red-tinted glasses.
The first half insist that the surface is blue, while the other half insist that it is red.
Finally they combine their perceptions and come to an agreement that the surface is purple.
But the surface is blank, it has no color. The alleged color of the surface was dependent on the perceptions and interpretations of people who were looking for color through a colored lens.
The hypothesis is not the hypothetical.
Your perception of the world is much the same. Everything you observe and interpret has been filtered through the lens of your beliefs, expectations and limitations. When you discuss a world that is alleged to exist outside of your beliefs, expectations and limitations you are assigning color to a blank surface.
The most prevalent modern belief systems, or lenses, exist as such that everything seen through them appears to ‘out there’.
Territory is the color of the lens we look through. Our beliefs, expectations and limitations are territory-tinted. An inescapable consequence is that things appear to be territory, not because we know they are, but because this is our way of seeing things.
To say that the map is not the territory is not enough. If there is territory outside of our territory-tinted view of it, we could not know, because we are unable to see anything that is not territory-tinted.
Like the blank surface that looks blue or red, or can be interpreted as purple, reality outside of our experience cannot be interacted with outside of our experience. There is no non-experience way of knowing reality; which makes the assumption in a reality outside of our experience entirely unverifiable.
The assumption that a territory exists outside of our maps is like saying that when nobody is looking at it with colored lenses, the blank surface is still either red, blue or purple.
Maybe there is a territory, but unless we figure out a way to transcend experience and interact with it in some currently unknown way, it is not a rational assumption to make. And yet all of our modern premises, scientific materialism, objectivism, etc. begin with just such an irrational assumption.