You are only speaking more in platitudes than specifics. I am actually very familiar with people of various relationships at many levels.
What is the gender of you and your spouse? Are you seeking a man or a woman?
What are you actually discussing? It could be everything from an occasional 3rd sexual playmate, to a mistress/mister on the side that you both share, or integrating a 3rd person into your marriage - which is polygamy, not polymous.
Here are short answers that will be accurate 90% of the time.
IF you both are pondering a 3rd lover, this must be 1.) that both of you are ALWAYS together with the 3rd person intimately and otherwise - absolutely NO exceptions and 2.) that this 3rd person NOT be a regular person and would have to be replaced over and over. If this 3rd person is the same person who share over and over together, that 3rd person will compete for one of you. The other one is greatly disadvantaged due to the mundane aspect of life.
IF you are looking to integrate someone in your marriage beyond sexuality, this is very, very complex. You seem to be thinking of this of only what happens if you cross the sexual fidelity barrier. But like all relationships of which many start on romantic and sexual attraction, the day to day things of life come to be of increasing importance. Integrating someone into your marriage is POLYGAMY, not polymorous - whether you want that or not.
Accordingly, any 3rd person you bring into your relationship is a really big deal. EVERYTHING has to be worked out - and in advance. What about money? Who pays what and how much? What about property? Who owns what? Who pays for what? Time allocation. Dispute resolution method. What happens to property if you break up. What about a child - what is the agreement then? - even if all of you SAY you don't want children?
For anything other than picking up temporary 3rd lovers only for intimacy that you both are ALWAYS together with, anything approaching a permanent integration should be literally written out, signed and notarized in advance. I can not understate how important I believe that is, though few actually do so.
ANY adding a 3rd person WILL lead to a power struggle. It WILL happen. Unless all of you have agreed in advance how that will be resolved, it will lead to a disaster.
IF you both are thinking about it more as crossing a sexual fidelity threshold for fun or interest or whatever, a good way to do that would be to hire someone for this the first time - literally a prostitute (male or female such as you are looking for) - and then going at that with both of you present. That way you both understand that it is neither of you seeking a new relationship because the 3rd person is just someone you both hired for a specific purpose only. There is less risk in that experimentation because there is no relationship possibility and it understood it only about sexuality. That is first "test" to see how you both feel about it - and about each other - afterwards.
Rarely do 3 way or polymorous relationships go the whole distance, but I think that is sometimes for lack of recognizing the complexities involved and not addresses those up front - and taking it one step at a time. The challenges and problems have to be decided before they exist. Again, the CORE rule is an agreement that at least at first, there is NEVER a minute that you both are not together with the 3rd person - unless it a notably rare type relationship. I could write about some of those, but those don't seem like they would apply.
Examples include where people in the relationship have differing sexual orientations and those are actual orientations. If the 3 are not the same gender, at least one of them has to be bi if all 3 are sexual in the relationship - though not necessarily as 3-somes. I could write out all manner of variables that come into play regarding sexual orientation prospects.
Rather than just generalities, what exactly are you and your spouse talking about? Is it just sexuality with a 3rd person? Or more? If so, how much more? STUMBLING along to see where it goes is all but certain disaster. Rather than just talking about it in philosophical terms, try talking about it in realistic, practice terms. THAT is when it starts getting complex.
Inherently you are on dangerous ground. Also, just a note, sometimes when someone in a relationship brings up the topic they are secretly actively interested in specifically someone else or are already involved, seeking prior-approval before or after the fact.