The fall of the Profile

 FireLy has looked at #FHIR use, and came to the conclusion "Too many profiles, not enough reuse...". I agree and find this trend very troubling.

IHE started the concept of Profiling 25 years ago. An joint effort of Vendors and Users. The Users would collaborate on use-case based needs, needs fully focused on outcomes and overcoming problems. The Users tempted the Vendors with a promise to "buy" if the Vendors agreed to One solution. Economics drove this to succeed.

Lately neither of these parties are leading, rather it is Governments and Consultants (yes I am now a consultant). This not only doesn't have the right Market forces, but is not done globally. with no global focus the solutions are regional.. all different.

AI use Transparency in Healthcare: Building Trust Through Provenance

I want to bring some additional visibility to a project I am involved in regarding AI transparency in Healthcare. The goal of Transparency is to be able to indicate when data in the Medical Record has been influenced by AI, this is an important goal to providing Integrity of the use of AI. 

The Challenge: A Spectrum of AI Influence

The goal of our project is to indicate the level of AI influence on medical data. This isn't a simple "yes or no" question, but a spectrum that includes:

• AI-authored data: The data was created entirely by an AI.

• AI-recommended data: An AI suggested the data, and a human approved it.

• AI-assisted data: An AI helped a human in some way, but the human was the primary author.

To address this, we're using two key approaches: data tagging and provenance.

Data Tagging

With data tagging, this is simply a tag of the kind of interaction that the AI had with a data object. So it is not useful to explain the details of the interaction beyond a generalizable kind of interaction. This tag is however helpful as a flag for those who want to know when data was influenced. 

One use of a simple tag is to recognize that the object may be not original thinking. There might be recognition that data that has been influenced by AI might not be as useful to train future models. The tag might also be used simply to know that there is more details in a Provenance.

Provenance

With Provenance we can carry details about what AI, what version, what model, what prompt, what card, etc. The FHIR Provenance is a derivative of W3C PROV, reformed to the data encoding standard that HL7 has based on RESTful Resources. 

We are trying to reuse more general AI standards such as model-card, but find that there is a lack of consensus. I am confident that the HL7 group will use external standards as appropriate.

One might need to know this level of detail to understand the usefulness of the output. One might also use this Provenance to track down AI influence that may have been determined to be suspect or incorrect. This might find decisions that need to be reevaluated.

Element level, not just Resource level

Both data tagging and Provenance have methods of focus on the element level, rather than the whole Resource. For some resources the whole resource is all that is needed to be tagged or referenced, but for some more workflow specific Resources like CarePlan, there are some data within that might be influenced while the whole is not. So, this element level is supported by both Data Tagging and Provenance solutions.

Concerns with Provenance model

A concern I heard was voiced at the connectathon this weekend is that Provenance is hard to work with. I think this is just an educational issue. Provenance is different in that Provenance.target points at the resources for which it is describing the provenance of; and thus the targeted resource does not contain some evidence of the Provenance. There are a few solutions to this:

1. Use the Data Tag to indicate that the data was influenced by AI, and this gives evidence that searching for Provenance might be useful. When the AI tag is found, one just searches for Provenance with a target equal to the resource you have.
2. Put the Provenance inside the Resource. FHIR supports a concept of a Resource "containing" another resource. This is used when the contained resource can't stand alone, but can also be used where the outer Resource really wants to carry the inner Resource
3. Searching for resources, one can use the "_revinclude" parameter to also include any Provenance. Indeed, _revinclude is defined for anything, but the example given is Provenance.

Developing Implementation Guide

The HL7 implementation guide is in development so I don't, yet, have a formal publication to point at. The CI build is -- https://build.fhir.org/ig/HL7/aitransparency-ig/branches/main/index.html

All of the above discussion is already included in this Implementation Guide.

I have other blog articles on AI controls 

Learning Dataset Provenance

Wearing a different hat, I was a standards expert contract with Data and Trust Alliance to help them define a Provenance standard for the datasets that are offered to be used as source-learning material. https://dataandtrustalliance.org/work/data-provenance-standards

Conclusion

These are developing, so please get involved to help us address your use-case and learn from your experience. 

Approach to Product use of Standards

I have expressed a role for me as a standards expert to participate with product development to assure good implementation. This would focus on quality implementation, that is robust, and can then stand the test of time. However, I really don't think that this is a standalone role, but rather a role that someone on the product team plays. Likely a systems architect, maybe the db architect.

Now that I have started my consulting organization, Moehrke Research LLC, I have been approached by people trying to get me to take on this kind of a full-time role. The role is rather consistently defined and defined in a very standalone way with what I think is unreasonable expectations. The Job description includes many years of standards work, many years of product development, many years of healthcare market knowledge, etc. Job titles like:

• FHIR (Fast Healthcare Interoperability Resources) Architect
• Lead Data Modeler (FHIR)
• FHIR Interoperability Specialist
• Senior IT Solutions Architect
• Healthcare Solution Architect

I fit these expectations, but I really don't think that what you need is full-time position. I think that it is a great medium sized engagement with me. 

I recommend Build from within

Where someone (or two) from the product team get elevated. Yes, it is an additional role and thus a change in their role. I assure you paying them a bit more to take on this role will be worth it. You need to include a test engineer as well. I work with them 2-3 days a week for a few months, then a few days a month for a few more months, and then a few hours per month for a few more months. Overall, this likely takes 6-9 months. I teach them how to:

• discover appropriate standards, 
• approaches to reading standards, 
• extracting the requirements and alternatives,
• where to find help, 
• where to find open-source,
• where to find test tools and procedures,
• how to leverage Postel's Law, 
• how to engage in improving the standard,
• how to dispute interpretations of the standard,
• where to get creative and 
• where to be strict. 

With an engagement like I am proposing, I am providing this guidance over 600-1000 hours; and you walk away with the skills on the team. This is a bargain relative to a full-time position for 6 months. We all have a personal relationship that can handle occasional contact or result in future contract engagements.

More sustainable

The roles that are posted are not possible to be met except by a few dozen people globally. The expectation of number of years of experience, depth of knowledge, and unusual education. There is simply not that many people doing what I have done over the past 25 years. It is very small group (I would like to see it expand). 

Interoperability is not something to build a product around, it is something to build a product on-top-of. Meaning it is not the inspiration for something that doesn't exist. The standard was written because many have needed something like what the standard has defined. 

There are HL7 training certifications that can help, but I see these also as something that someone already on your team adds to their job roles.

Conclusion

Similar is true of the other topic areas I have skills in: Privacy and Security... these are a role, but not necessarily a full-time position. These are all more a culture thing, with a role to watch that the culture is followed.

In very large organizations like Oracle Health, Epic, GE Healthcare, etc... these can be full-time roles; but even there are constant struggles with justifying standalone positions. Even in these large organizations the sustainable position is a role that team members take on.

Build your team from within. I provide subject matter expertise, but your team is key. We all walk away happy and with better Interoperability.

Product use of Standards

The third kind of contract I’m well-suited for involves working directly with product developers—whether client or server-side—to ensure their solutions optimally leverage existing standards. This role is often overlooked in traditional standards development but is critical for real-world adoption. While it may seem like a large engagement, it often resembles a small contracts, focused contract. I’ll explore that nuance more in the next blog post.

Government Mandated Standards

A product can be compelled to be compliant with a standard or Implementation Guide (IG). This is common nowadays around the globe with regulation requiring that products and the organizations that use them be compliant with a given standard or IGs. These government efforts are trying to move their realm beyond some point, with the goal of having a better outcome after the standards are deployed.

A good example of how a government required standard can dramatically improve that realm that government controls is electric socket, light socket, or lately USB-C. In these cases, without standardization there were many alternatives that the consumer must be burdened with. By mandating a standard, the products all align on that standard, the consumers don't need to think about that anymore.

Purchase Power

A product may choose to implement a standard because market pressures (customers) demand it. In this case it is the power of the purchase ($$$) that forces the use of a standard. An important perspective here is where the first vendor works with the purchaser to define that which all later must implement. In the case of early Health Information Exchanges, and Radiology Exchanges; this was the dominant method for standards to become required. That is to say those purchasing products demanded that a given IHE-Profile must be used, and that drove mandates. This was the success story for IHE, in that it was a collaboration between those with purchase power in the radiology departments that wanted ONE standard to mandate, and the vendors that knew that one standard would be less overall work. Unfortunately, this story has been lost to time

Implement Once, Innovate Beyond

The overall benefit to using standards that those developing products that need that standard can now be assured that their efforts to use the standard will be reusable over and over; thus, that product development group can focus more on the features and value of the product. A good standard is one that one can implement once and not spend more time on (realistically it takes some maturing to get here). The point is that by using a standard one does not need to constantly adjust how one communicates with peers. 

This use of standards overall benefits everyone. 

What help do you need?

The fact that a regulation picks a standard or IG does not mean that developing a product to that standard is easy. The standard might not be all that easy to read, most standards are hard to read. The needs that the product have might not be expressed in the standard, so some interpolation needs to be done. There are often things that are needed to be implemented that the standard doesn't mention, most of the time it is in an area where the standard wants to be lenient, so one must understand a range of possibilities.

Postel's Law

When I work with your team, I will stress multiple times a day a principle that is credited for the success of the TCP/IP internet. Often called Postel's Law. It has two very different things to say. To those about to send some content to another, be as compliant as you possibly can be. To those receiving some content from another, be very robust and lenient in how you interpret that content. Many people have a problem with this second part as they feel that receivers should be strict, rejecting anything that is not compliant. The problem with this is that it is very fragile and doesn't recognize reality. Reality that often comes along with revisions of the standard over time.

Conclusion

Let me be your thoughtful, experienced guide in the often murky world of standards implementation.