It's simple. There's no justice in the first man not working while able while the second man does all the work just to survive. Just price is real.
One of the problems I find when people are asking what is right is that they assume unconstrained system. The thinking often goes "if X is right, then it always right no matter what and if Y is wrong, it is always wrong" This is because people tend to assume morality is based on reason or logic.
Instead, I would propose we take a page from psychology and see maslow's heirarchy of needs.
Then we combine it with two insights.
1. Moral foundations theory by Jonathan Heidt (the insights I am using here are just basic evolutionary psychology though)
2. John Hume's insight that morality is based on sentiment.
Combine them and you get something like this. Morality is a set of psychological drives aimed at social behavior as it applies to survival against either the wild or rival human groups (haidt). Morality is often based on perspective and sentiment (hume) and a human cannot consistently take their personal needs outside of their consideration for moral perspectives (again, hume). If you look at Maslow's chart, one's moral perspective is going to greatly depend on where one is in that chart. So if the late island arriver has their physiological needs threatened then their moral reasoning is going to start from that situation and what they do in order to survive against a rival group (in this case the groups are 1 vs 1, but there are still opposing purposes, thus creating the rivalry and triggering group psychology and associated instincts) is going to seem just. The same is true for the early arriver since they are protecting their place on the pyramid (probably rung 2 in this case since the other ones depend on being a part of a tribe or bigger group) and their need to preserve their land is going to seem right from their perspective.
So you get two people in a rivalry who both think what they are doing is just. There can be no social contract in that situation unless they find a reason to negotiate, but that would be hard to do since I suspect this is a very divisive topic as both see the situation as closely linked to their survival (which then triggers the whole psychological drive thing per haidt). However, if they are able to do that, then a new social contract is forged and hopefully they stick to it and things are good.
So, you cannot stuff a logical "x is right, y is wrong" in a situation like that, at least in any way I can think of, unless there is some sort of objective and morally perfect outside observer because personal and societal judgements cannot apply due to a complete lack of shared context (society is broken in this situation and its two people who think they are right and justified). This leads to the law of the jungle unless some outside force intervenes and applies their view of justice.
The other thing to think about is this is why love (agape type) and empathy are so important in life. It potentially bypasses all of this.
