Scripture never speaks about the Body of Believers as a 'church', but an assembly (ekklesia) and a koinonia (fellowship). This assembly/fellowship or ekklesia/koinonia of the Saved is the Body of Christ. We were created to be connected and assemble in some of the closest ways possible, and this includes within our own families and with other believers, even in our own houses. Yah'shua (Jesus) is our foundation and the corner stone that we are being built up on, so we may become a beautiful and qadosh (holy, set-apart) temple to house Elohim's (God's) presence. Authority such as is claimed by the Roman Catholic Pope, the Mormon President/Prophet, Jehovah's Witnesses' presiding body, etc. is therefore based on a misinterpretation and misapplication of Scripture.
When the Ruach (Holy Spirit) moved Paul to tell us that we are the Body of Messiah, it was not to say that Yahweh's Kingdom had no visible structure or order. Rather, the point that was being made is that we are not only connected in and through one Master and Head (namely Yah'shua/Jesus) but exactly that we ARE to be the visible, solid, ordered manifestation of Yahweh's Kingdom, through which those outside could experience His Character, His will and His touch – that the same way someone from the outside cannot see a person's invisible spirit (or mind or emotions) but can see a person's body, and can learn to 'see' and connect with that person's unseen spirit, mind, will and emotions through seeing what that person does, shares and says through their body. In the same way, the world is meant to be able to see and learn to know the Invisible Elohim (God), who is Ruach (Spirit), by seeing and hearing Him through us as we move together, submitted to His will, in love.
We know, just by seeing or looking at people's physical bodies, there is order and structure in the way they work, something that can be seen from the outside. We see some tissues, limbs and organs are designed to submit and respond to other tissues and organs (like the nervous system or glands for example) as they respond and transmit the message they were given from the Head. (And indeed, we know some of the most terrible suffering and disability in the physical comes when parts of our bodies lose connection or stop responding to other parts of our body the way they were designed to).
The concept of all righteousness being only 'spirit[ual]' or invisible, and that there is no visible organisation or connection in the assembly of the Saved isn't a scriptural one, but Gnostic. And indeed, multiple times throughout the New Testament, the testimony and call is (and was) that the believers who make up the Body Of Messiah not only be joined together in love in Messiah, BUT to submit to the authority and structure Elohim (God) Himself has designed and put in place, to respond and submit to the messages He gives through His ordained authority, so we can operate powerfully together, to His glory.
(This is paralleled by when Paul was moved to call us soldiers in the good fight, for soldiers not only fight together and have each other's backs, but they submit to the authority structure and authority of other higher ranking soldiers as they move on the authority of the Grand General of the Army).
If we reject all structure and having to be accountable to other human beings in Messiah, we will be part of a sickness or disability in the Body of Messiah – where part of the body won't listen to another operating under the authority of the Head.
Yes, MANY churches and denominations are exclusive and arrogant, the creations of, and for, mens' own prideful idolatries rather than to truly honour and serve Messiah, but we can't reject all structures and assemblies because the enemy has persuaded men to build counterfeits. Indeed, why counterfeit anything that has no worth? (When someone counterfeits money or fine art, isn't it exactly because the real thing has value and influence? Or when con-men pretend to be people of stations or stature they are not, isn't it because they are hoping to deceive, steal and abuse an authority that was someone operating in that true capacity would have used to serve for the good in?) The enemy has made counterfeits assemblies and structures exactly because there is power in ordered authority structures which can be used either for the good or bad.
As far as churches or denominations being inflexible about certain things, we need to understand and expect that even in Yahweh's true pattern and structure, there will be some things that will be inflexible, because there are some things Yahweh is clear is and isn't OK. In the same way a Body needs rigid bones to give it structure, and ligaments and tendons that put certain boundaries on how muscles and joints in a Body can and can't move, Father Yah's structure and pattern does have certain boundaries and strictness that is needed in order for us to function the way He intends us to.
We should also be honest with ourselves that our flesh wants to be elohim (god) – that in our flesh, we want to call all the shots. We don't want to have to be accountable to anyone, or have anyone telling us what to do. Being accountable to someone in a rightful authority position means having to be humble, that we can't always have our own way, that someone can reprimand us or require things from us we may not want to give. It's easier to ignore or belittle or argue away things that come from people we assert have no right or authority than with those we acknowledge do have true authority in our lives. That is at the heart of our rebellion against Yahweh, of children against their parents, of wives towards their husbands – of all rebellion against godly authority.
How easy and tempting is it then for our flesh and our fears to make excuses and say 'I will submit to Christ, but no man', that we can claim we are following Yah'shua (Jesus) but not have people who can call us to account? If we say we will submit to Christ, and Christ sends a man (and authority structure) that is under His authority into our lives, for us to be accountable to in Him (for our own good and for the purpose of an ordered, healthy functioning Body), and we refuse to submit to them, how can we claim to be submitted to Messiah who commanded us to submit that person or structure in the first place?
No, we shouldn't submit to just any church or teacher or person coming to us claiming to have Messiah's authority. We should be like the Berean Judahites, who tested everything in the Light of Scripture and the Ruach (Spirit), and when they found it to be true according to Scripture and the Ruach (Spirit) bore witness that Paul was truly speaking truth in Elohim's (God's) authority, received it and submitted gladly, because they understood they were ultimately not submitting mainly to the messenger (Paul), but Elohim (God) whose message and authority Paul was carrying.
We should be sensitive to both search the Scriptures to check what is true and to listen to the Ruach's leading, and not to harden our hearts against Elohim (God) when He calls us join a certain group or submit ourselves to a godly, Messiah-ordained authority in our lives, because we don't want to be accountable to others. We should be careful and discerning, but also sensitive and obedient when we do have the testimony from the Ruach (Spirit).
And if we feel Yahweh has called us to be a part and submit in a certain part of the Body, and we feel things aren't right yet, we should be honestly and humbly seeking to speak with those He has placed in authority over us, and not simply reject everything because they (or the structure) is imperfect. A Body needs to give feedback in order to adjust. We are supposed, in love and respect, to be seeking perfection together in Him!
When we find and submit ourselves to what Yahweh Himself has ordained for us, its not only for our own good and blessing, but the blessing of the whole Body, and the great beauty and privilege of being able to know and experience and express Father Yah's will and His way. Let's not miss that!