AI use Transparency in Healthcare: Building Trust Through Provenance

I want to bring some additional visibility to a project I am involved in regarding AI transparency in Healthcare. The goal of Transparency is to be able to indicate when data in the Medical Record has been influenced by AI, this is an important goal to providing Integrity of the use of AI. 

The Challenge: A Spectrum of AI Influence

The goal of our project is to indicate the level of AI influence on medical data. This isn't a simple "yes or no" question, but a spectrum that includes:

• AI-authored data: The data was created entirely by an AI.

• AI-recommended data: An AI suggested the data, and a human approved it.

• AI-assisted data: An AI helped a human in some way, but the human was the primary author.

To address this, we're using two key approaches: data tagging and provenance.

Data Tagging

With data tagging, this is simply a tag of the kind of interaction that the AI had with a data object. So it is not useful to explain the details of the interaction beyond a generalizable kind of interaction. This tag is however helpful as a flag for those who want to know when data was influenced. 

One use of a simple tag is to recognize that the object may be not original thinking. There might be recognition that data that has been influenced by AI might not be as useful to train future models. The tag might also be used simply to know that there is more details in a Provenance.

Provenance

With Provenance we can carry details about what AI, what version, what model, what prompt, what card, etc. The FHIR Provenance is a derivative of W3C PROV, reformed to the data encoding standard that HL7 has based on RESTful Resources. 

We are trying to reuse more general AI standards such as model-card, but find that there is a lack of consensus. I am confident that the HL7 group will use external standards as appropriate.

One might need to know this level of detail to understand the usefulness of the output. One might also use this Provenance to track down AI influence that may have been determined to be suspect or incorrect. This might find decisions that need to be reevaluated.

Element level, not just Resource level

Both data tagging and Provenance have methods of focus on the element level, rather than the whole Resource. For some resources the whole resource is all that is needed to be tagged or referenced, but for some more workflow specific Resources like CarePlan, there are some data within that might be influenced while the whole is not. So, this element level is supported by both Data Tagging and Provenance solutions.

Concerns with Provenance model

A concern I heard was voiced at the connectathon this weekend is that Provenance is hard to work with. I think this is just an educational issue. Provenance is different in that Provenance.target points at the resources for which it is describing the provenance of; and thus the targeted resource does not contain some evidence of the Provenance. There are a few solutions to this:

1. Use the Data Tag to indicate that the data was influenced by AI, and this gives evidence that searching for Provenance might be useful. When the AI tag is found, one just searches for Provenance with a target equal to the resource you have.
2. Put the Provenance inside the Resource. FHIR supports a concept of a Resource "containing" another resource. This is used when the contained resource can't stand alone, but can also be used where the outer Resource really wants to carry the inner Resource
3. Searching for resources, one can use the "_revinclude" parameter to also include any Provenance. Indeed, _revinclude is defined for anything, but the example given is Provenance.

Developing Implementation Guide

The HL7 implementation guide is in development so I don't, yet, have a formal publication to point at. The CI build is -- https://build.fhir.org/ig/HL7/aitransparency-ig/branches/main/index.html

All of the above discussion is already included in this Implementation Guide.

I have other blog articles on AI controls 

Learning Dataset Provenance

Wearing a different hat, I was a standards expert contract with Data and Trust Alliance to help them define a Provenance standard for the datasets that are offered to be used as source-learning material. https://dataandtrustalliance.org/work/data-provenance-standards

Conclusion

These are developing, so please get involved to help us address your use-case and learn from your experience. 

Approach to Product use of Standards

I have expressed a role for me as a standards expert to participate with product development to assure good implementation. This would focus on quality implementation, that is robust, and can then stand the test of time. However, I really don't think that this is a standalone role, but rather a role that someone on the product team plays. Likely a systems architect, maybe the db architect.

Now that I have started my consulting organization, Moehrke Research LLC, I have been approached by people trying to get me to take on this kind of a full-time role. The role is rather consistently defined and defined in a very standalone way with what I think is unreasonable expectations. The Job description includes many years of standards work, many years of product development, many years of healthcare market knowledge, etc. Job titles like:

• FHIR (Fast Healthcare Interoperability Resources) Architect
• Lead Data Modeler (FHIR)
• FHIR Interoperability Specialist
• Senior IT Solutions Architect
• Healthcare Solution Architect

I fit these expectations, but I really don't think that what you need is full-time position. I think that it is a great medium sized engagement with me. 

I recommend Build from within

Where someone (or two) from the product team get elevated. Yes, it is an additional role and thus a change in their role. I assure you paying them a bit more to take on this role will be worth it. You need to include a test engineer as well. I work with them 2-3 days a week for a few months, then a few days a month for a few more months, and then a few hours per month for a few more months. Overall, this likely takes 6-9 months. I teach them how to:

• discover appropriate standards, 
• approaches to reading standards, 
• extracting the requirements and alternatives,
• where to find help, 
• where to find open-source,
• where to find test tools and procedures,
• how to leverage Postel's Law, 
• how to engage in improving the standard,
• how to dispute interpretations of the standard,
• where to get creative and 
• where to be strict. 

With an engagement like I am proposing, I am providing this guidance over 600-1000 hours; and you walk away with the skills on the team. This is a bargain relative to a full-time position for 6 months. We all have a personal relationship that can handle occasional contact or result in future contract engagements.

More sustainable

The roles that are posted are not possible to be met except by a few dozen people globally. The expectation of number of years of experience, depth of knowledge, and unusual education. There is simply not that many people doing what I have done over the past 25 years. It is very small group (I would like to see it expand). 

Interoperability is not something to build a product around, it is something to build a product on-top-of. Meaning it is not the inspiration for something that doesn't exist. The standard was written because many have needed something like what the standard has defined. 

There are HL7 training certifications that can help, but I see these also as something that someone already on your team adds to their job roles.

Conclusion

Similar is true of the other topic areas I have skills in: Privacy and Security... these are a role, but not necessarily a full-time position. These are all more a culture thing, with a role to watch that the culture is followed.

In very large organizations like Oracle Health, Epic, GE Healthcare, etc... these can be full-time roles; but even there are constant struggles with justifying standalone positions. Even in these large organizations the sustainable position is a role that team members take on.

Build your team from within. I provide subject matter expertise, but your team is key. We all walk away happy and with better Interoperability.

Product use of Standards

The third kind of contract I’m well-suited for involves working directly with product developers—whether client or server-side—to ensure their solutions optimally leverage existing standards. This role is often overlooked in traditional standards development but is critical for real-world adoption. While it may seem like a large engagement, it often resembles a small contracts, focused contract. I’ll explore that nuance more in the next blog post.

Government Mandated Standards

A product can be compelled to be compliant with a standard or Implementation Guide (IG). This is common nowadays around the globe with regulation requiring that products and the organizations that use them be compliant with a given standard or IGs. These government efforts are trying to move their realm beyond some point, with the goal of having a better outcome after the standards are deployed.

A good example of how a government required standard can dramatically improve that realm that government controls is electric socket, light socket, or lately USB-C. In these cases, without standardization there were many alternatives that the consumer must be burdened with. By mandating a standard, the products all align on that standard, the consumers don't need to think about that anymore.

Purchase Power

A product may choose to implement a standard because market pressures (customers) demand it. In this case it is the power of the purchase ($$$) that forces the use of a standard. An important perspective here is where the first vendor works with the purchaser to define that which all later must implement. In the case of early Health Information Exchanges, and Radiology Exchanges; this was the dominant method for standards to become required. That is to say those purchasing products demanded that a given IHE-Profile must be used, and that drove mandates. This was the success story for IHE, in that it was a collaboration between those with purchase power in the radiology departments that wanted ONE standard to mandate, and the vendors that knew that one standard would be less overall work. Unfortunately, this story has been lost to time

Implement Once, Innovate Beyond

The overall benefit to using standards that those developing products that need that standard can now be assured that their efforts to use the standard will be reusable over and over; thus, that product development group can focus more on the features and value of the product. A good standard is one that one can implement once and not spend more time on (realistically it takes some maturing to get here). The point is that by using a standard one does not need to constantly adjust how one communicates with peers. 

This use of standards overall benefits everyone. 

What help do you need?

The fact that a regulation picks a standard or IG does not mean that developing a product to that standard is easy. The standard might not be all that easy to read, most standards are hard to read. The needs that the product have might not be expressed in the standard, so some interpolation needs to be done. There are often things that are needed to be implemented that the standard doesn't mention, most of the time it is in an area where the standard wants to be lenient, so one must understand a range of possibilities.

Postel's Law

When I work with your team, I will stress multiple times a day a principle that is credited for the success of the TCP/IP internet. Often called Postel's Law. It has two very different things to say. To those about to send some content to another, be as compliant as you possibly can be. To those receiving some content from another, be very robust and lenient in how you interpret that content. Many people have a problem with this second part as they feel that receivers should be strict, rejecting anything that is not compliant. The problem with this is that it is very fragile and doesn't recognize reality. Reality that often comes along with revisions of the standard over time.

Conclusion

Let me be your thoughtful, experienced guide in the often murky world of standards implementation. 

Standards Development Contracts

The medium sized contracts that I envision would be where I help an organization develop a standard or defend their position within a standard development project. Where a "standard development" effort is not limited to core standards like FHIR, CDA, or HL7; but inclusive of international Implementation Guides (what IHE calls a Profile, or HL7 calls accelerators), or regional Implementation Guides. 

I have even used the Implementation Guide tooling to produce a private publication (for the VA - MyHealtheVet) that defines how existing data would map to FHIR Resources and be aligned with us-core. I use this tooling in my own experiments as it is a quick way to get a publication that is easy to author and edit over time. 

Leading a standard project takes a good bit of negotiations and consensus building. These are skills that I have been working over the past 25 years (actually more, as I also did this in the internet standards world in the 80s and 90s with TCP/IP, NFS, Telnet, FTP, and a few others that many people today don't remember are foundations of the internet.)

Defending an organizations position is similar but very different. It involves discovering the potential problems and crafting a solution that the author and contributors find as understandable and worthy of addressing. Sometimes this effort is simply helping by providing examples of good and bad outcomes; such as working examples.

Along with this is providing tooling to support internal testing, simulation, and demonstration.

Developing Standards is the best way to develop a market for your product to further enhance. Standards are not a threat to a product unless that product is not truly adding value. By defining standards, one moves the opportunity for improvement up into the application layer.

Organizations, which might be provider or payer organizations, or might be regional organizations, often need to refine a standard to make it more clear for their region, and thus make testing and dispute resolution more effective.

I am well-seasoned to be able to help you with this effort. These projects might be medium, but they might also be small or large. The size is more defined by the outcome needed. Contact me at Moehrke Research.

Small contracts are also encouraged

Small contracts

Over the past few years, I have taken on small contracts. These would be a few hours and be focused on delivering a training session or two. These were never big enough projects for my employer at the time, so they allowed me to take these on "the side". I would tend to work these in the evenings and weekends so as to not interfere with my day job at the time.

Now that I am looking for contracts, these small contracts are something I am looking forward to. There is not much fuss in getting them going, and they are a great way for me to interact with groups of people just getting going in Interoperability or Healthcare Informatics. 

Training in Healthcare Privacy and Security

The subject matter that I am known for is teaching FHIR Privacy and Security topics. I have presented a HL7 Tutorial on "FHIR Privacy and Security" many times. I am not limited to giving this tutorial at HL7. HL7 has a recording from a few years ago that is freely available through HL7 sponsored by ONC (now ASTP). If you just want to listen to the recording, then the HL7 recorded tutorial is good enough. But if you have specific use-case that you want me to focus on and have discussion, design, and policy writing; then this might be a good small contract to start with me.

I can also handle going deeper on each of the topics within the tutorial. I have had to make the 3-hour tutorial very high level. Which is a good level for many people, but does not satisfy someone who is focusing on a given topic:

• Access Control - considering Privacy Consent
• Access Control - considering Break-Glass
• Audit Logging - to detect intrusion and investigate
• Audit Logging - to inform an Accounting of Disclosures or Access Log to a Patient
• Digital Signatures
• Document Encryption
• Consent encoding in FHIR and management over time
• Data Sensitivity Tagging methodologies and architectures
• De-Identification / Pseudonymization / Anonymization
• Provenance

Training in Healthcare Infrastructure -- Implementation Guides

• IHE IT Infrastructure Profiles
• XDS / XCA / XCPD -- Document Sharing
• MHD / MHDS / PDQm / PMIR
• mXDE -- decomposing Documents into FHIR Resources with Provenance
• Basic Audit Log Patterns (BALP)
• Privacy Consent on FHIR (PCF)
• Digital Signatures (DSG)
• HL7
• FHIR International Patient Summary (IPS)
• FHIR International Patient Access (IPA)
• FHIR Data Segmentation for Privacy (DS4P)
• FHIR Consent
• FHIR AuditEvent
• FHIR Provenance
• FHIR Signature

Contact me at Moehrke Research.

What's next for me?

It has been a relaxing week, but I am still interested in opportunities for me. I have had a handful of phone calls. I hear that my name is mentioned positively in many conversations, that I am not involved in.  LinkedIn tells me that my announcement has been seen 11,000 times, my blog article only 104. So, you can understand that I am getting really dramatically mixed messages.

What I'm looking for

I have put together a Resume, and doing that did solidify my interests in 

1. Standards development (FHIR)
2. Profile development (Implementation Guides)
3. Use of Profiles and Standards (Apps and Infrastructure that use standards)

I don't like doing the administrative things that a consultant needs to do, like finding new work, billing, following up on billing. I did this a few times over the last few years, and it is outright drudgery. I also don't want to move into a corporate position, like a director of blah, or senior so-and-so. Learning a new corporate process is not inviting, I did that three times already.

I know that I am close to retirement, I can feel the beach sand beneath my feet.  Thus, I understand that whatever I do needs to make this transparently clear. And I recognize that might limit my opportunities. I am okay with that.

Ongoing Contributions

I will continue my work in HL7 and SHIFT:

• SHIFT on their work to make Consent more implemented. I have been providing subject matter expertise in FHIR Consent and the IHE Privacy Consent on FHIR (PCF) implementation guide. Right now, the team is working on implementing, so I don't have much to contribute. I would like to be involved in code reviews. I am also providing expertise in the discussion with various stakeholders and implementers. 
• HL7 on their work with FAST Consent, which is taking an administrative step beyond IHE-PCF to define policies and management steps for instances of Consent. This work can only be done in a regional context where regional policies can limit the variability. So, context is critical here. Having reviewed many regional policies and applied them to the development of FHIR Consent, I have a pragmatic and realistic perspective to provide.
• HL7 on AI Transparency IG, which is using features we built into FHIR for tagging data that was contributed by AI and providing details of that AI actions in Provenance.  I have applied these concepts to IHE profiles and Data Trust Alliance, and other side projects. The power of Provenance is best shown with use-case analysis and examples. 

I will continue with these, even if only Pro Bono. I certainly hope that I have other opportunities that I could contribute to. I think I still have plenty of energy and expertise to be applied to Healthcare Interoperability Standards development and promulgation.

Monday Morning, nowhere to report

This Monday morning started differently—I’m awake, ready to work, but with nowhere to report. 

After nearly nine years at ByLight, my journey there has abruptly ended. ByLight had taken me on as a standards representative back in November of 2016. I have worked on multiple CDA, XDS, and FHIR-based projects ever since. Most recently, I was helping modernize MyHealtheVet—the VA’s patient portal where Veterans securely access their medical records, message their care teams, and manage prescriptions. Our team was about halfway through a FHIR transition and updating the web interface, with Oracle Health (Cerner) integration just beginning to support the VA’s evolving EHR ecosystem.

But the contract was unexpectedly not renewed. We were all let go, and I imagine many of my colleagues are now, like me, seeking what comes next. ByLight fought hard to continue the work, but I don’t know what led to the decision or what the future holds for the portal itself.

What now?

I had expected to retire somewhere in the next 2–5 years, with time to prepare and transition. That plan changed overnight.

Today, I was supposed to co-chair the IHE IT Infrastructure Technical Committee’s face-to-face meeting. Instead, I had to inform my co-chairs and peers that I’m no longer employed and no longer have standing within IHE. Others had scheduling challenges too, so we opted to postpone and shift to our regular t-con development calls instead. IHE will also need to redistribute the roles I held—GitHub administration, IG publishing, and more.

I’ve also informed HL7, and they’ve revoked my authority. I recall from before that HL7 has a method to extend membership continuity, but I haven’t heard whether that will apply to my case.

What's next?

While I had begun thinking about retirement, this came too soon. I’m now exploring consulting work—perhaps independently, perhaps through a contracting organization. I’m not interested in stepping into a dramatically different role or climbing further up the leadership ladder; when I look at what’s done “up there,” I don’t find much that sparks inspiration. It doesn’t feel like the right way to wind down. More likely that I use my FHIR Implementation Guide experience and skills to help projects and regions on their profiling.

I’ve seen other standards geeks continue consulting into their later years, which I’ve always viewed with mixed feelings. The world needs space for fresh leadership, and that’s hard to foster when the same people continue to occupy those positions.

So, here I am again—like I was in 2016—dusting off my resume and pondering the next chapter. I do have a few camping vacations scheduled from before all this, and I plan to take them. Maybe the timing will turn out to be fortuitous.