Using Mutual-Authenticated TLS
This is easy to specify, and is consistent with IHE-ATNA.
This solves authentication of the server to the client, and authentication of the client to the server.
This solves the encryption and data integrity (authenticity) problem.
Thus keeping EVERYONE else on the internet out of the conversation.
The negative of this is that one must manage Certificates. One issued to the Client, One issued to the Server. The more clients and servers you have, the harder the management of these Certificates become. As this number approaches a large number (greater than 2 by some peoples math, greater than 20 by others) it becomes more manageable to involve a Certificate Authority. You can use a Certificate Authority from the start, but it is not that critical. Some Operational-Security-Geeks will demand it, but often they are misguided..
So at this point we have a simple solution, that addresses the issues. It looks very good on paper.
Problem with Client authenticated TLS
The next problem is that the TLS acceleration box strips off TLS; meaning that my web-server gets no indication of the identity of the client. If the situation is that I don't have different Access Control rules, this might not be a big problem. However I don't have a way to record in the Audit Log who the client is. If the client is exactly ONE system, then I can guess that it is that system.
The good news is that the TLS acceleration box can likely be configured to pass along that client identity from that TLS client authentication. In my case, there was a chapter in the F5 documentation that told me how to write the script to be inserted in the F5 so that it would extract the Client Identity, and stuff it into a http header of my choosing. Thus my web server could look at that header for the identity of the client. Of course I had to make sure that the header NEVER came from the external world, or it wouldn't be an indication of a authenticated client. This is a kludge, but a defendable one.
Using OAuth is better?
The advantage of this is that the OAuth token gets forwarded directly to your web stack, where it can be used for Access Control and Audit Logging. It can be verified, based on OAuth protocol (and all the security considerations left open by OAuth). Really nice for both service orchestration and 'scale' is that it can be further forwarded to sub-services which can validate it, use it for access control, and audit logging. This is really important feature if you have a nice modular service oriented architecture.
The drawback of OAuth is that you must include an OAuth authority in the real-time conversation. Where as with Mutual-Authenticate-TLS, the certificate is issued once and good for a couple of years; the OAuth token is often only good for 24 hours, or 8 hours, or less. You could issue 2 year OAuth tokens, but that is unusual.
I need to note that in order to use OAuth you do need to deal with registering the client 'application'; which is often done via a static secret (password like), or via a certificate... Everything ultimately does come down to a secret or certificate. I would recommend certificate, but realize that is not the common solution. Most implement only secret key.
Both is better?Not really. The only benefit you get by using both Mutually-Authenticate-TLS and OAuth is that the hardware accelerated TLS box (F5) can reject bad clients with less overhead on your web-stack. This is a benefit, but you need to weigh this against the cost of certificate issuing and management.
It is however better in that it has the fewest hacks to get it to work fully.