Monday, September 17, 2018

My reportout on ONC Interop Forum panel discussion

I have not written a blog article on the output of the Panel discussion I lead at the ONC Interop Forum. The discussion I wrote up before the event is a good place to start. The panel, as one would expect, did deviate slightly with a much more focus on:

  1. Difficulty implementing Privacy controls with the large number of privacy regulations. This complexity becomes very difficult around the edge cases: Going across state lines, Emancipated Minor, Minor transition, Elderly delegated access, etc. The panel members were NOT looking for less privacy rights, just more clear rights. 
    • There is some benefit to federation level overrides such as we now see with Europe GDPR
  2. Provenance of the data submitted by the Patient. Where did the Patient get the data? How do we know the Patient hasn't changed the data? Most Patients are good, but some are being nefarious, for example drug seeking behavior. There is a need for clear guidance on minimal Provenance information.
  3. The four modes of communication became more clear as well as a model to discuss the interactions with the Patient.
  4. Ask to ONC, from Sean during my panel "Please help make the right thing to do, the easiest thing to do." Simplify Privacy complexity without removing privacy rights. 
  5. Patient population is not uniform. There are some Patients that want to be very actively involved in the flow of their data, there are many other Patients that want their data to be readily available for Treatment purposes without them needing to get involved.

Luck for me the day 1 and day 3 reports are available on YouTube now, published by ONC. There are two recordings of significant length. Each minute is useful to listen to.

See the video of me reporting out on the third day just prior to hour 2 of the 6 hour recording available. The following video should start right at the beginning of that report out (2:16:00). Additional action items for ONC from my panel at 2:36:15. Response to Adrian Gropper question at 2:43:29 regarding patients activity.



Brand new sport coat... How do I look?

Friday, September 7, 2018

Webinars on MHD and mXDE available from IHE


 I presented these same slides at the FHIR DevDays. The overall goal is 15 Minute presentations that build first explaining general Document Sharing (XDS, XCA, etc), then Mobile access to Health Documents (MHD), and finally the Mobile Cross-Enterprise Document Data Element Extraction (mXDE) which orchestrates Query Element Data for Mobile (QEDm).

The slide deck is also available to View and download.

View Document Sharing Slides




View MHD Slides



















View mXDE Slides



Improving Document Exchange Response with Asynchronous FHIR API

In the last article I show how PDQm can be made Asynchronous by using the FHIR Subscription. This is just one step in the overall interaction that one does with a Cross-Community Document Exchange (XCA). It is however the one that typically has the largest delay.

Baseline using Synchronous API

Here is a simplified view of a Consumer requesting through a synchronous API to do a Patient Discovery (PD), Query for Documents (QD), and Retrieve Documents (RD). A more detailed view in previous articles. This high-level view works for MHD as an API to XCA, just as well as using XCA/XDS old-school.

Lets imagine there is 200 partners within the exchange. The Patient Discovery must be sent to each one of them. This fan-out is not shown, it is being done by the Service on the right. The Service must wait for the response to come back from each of them. Only when all partners have responded, or some timeout has happened, can the results be returned back to the Consumer.  As I indicated before some of the 200 partners might take a few minutes to return a result. Thus this Patient Discovery can take minutes before the second step of Query for Documents can happen.

The Query for Documents is a bit more targeted, as you would only query the partners for which you got a positive patient identity match. In the example from 200 partners, only 10 of them know of the patient of interest. Thus the Query for Documents can be limited to just those 10. It is still possible that those 10 are slow.

The Query for Documents to those 10  can be done in parallel if your Consumer technology stack allows that. I show it here as 10 one after another. Thus the overall Query for Documents will be the combination of all the partners including the slowest partner that has data on the patient of interest.

This diagram shows that for every Document Entry found, a Retrieve Document will be done. This is not unusual for some middleware that might be decomposing those documents, such as the mXDE environment I show. I guess I really need to blog that mXDE environment.

Also there are opportunities to detect the same document at multiple locations, thus eliminating the Retrieve Document being done multiple times. See this article.

Better Response using Async API

There is asynchronous support in XCPD/XCA; it is just not often used. For example the eHealth Exchange and CareQuality don't have any use of asynchronous while they have required everyone to support it.  I understand that the Epic Care Everywhere internal network that is based on XCPD/XCA does use the Asynchronous SOAP. Using Asynchronous SOAP gets the same improvement shown here.

This improvement can also be done using FHIR Subscription and orchestration of each notification. The diagram shows one full Patient Discovery, Query for Documents, and Retrieve Documents. The last notes shown just indicate the same could be done for each result returned.

Something that is hard to diagram is that all of these events are asynchronous so everything is overlapping in parallel.

The result is that the first full Document returned might be available in fractions of a second, while the last might take a long time. In fact the last one would only take as long as the three steps (PD+QD+RD) for that entry alone. This Asynchronous approach decouples the response of each partner.

Conclusion

Async is great... but it still must be managed, and the failure-modes are messy. If one has infinite resources, for example an infinite number of threads, then one can just let the system go. But often one has just a limited number of threads or memory or cache.. Thus some events might need to be queued to happen when resources come available.

Automating the Document Retrieve is not as important, but showing it can be orchestrated in just hammers home the message. This step would not be done for human view, but would be done for a middleware service that is decomposing the documents, like mXDE describes.

Even more advanced orchestration

The other use-case is the one that David Vazquez and I worked together on as an internal API within the VA. It is David that made this vision come to life. He orchestrated PD->QD->RD completely. He made an internal API so had some design criteria to help him.  Used FHIR Subscription on DocumentReference. There was clear Patient search parameters that can be chained cleanly onto DocumentReference query. There is a need for cache of DocumentReference and Documents. By having this cache could then respond to requests from the app quickly.  He then changed the Web based application the Clinicians use to use this Async MHD like API, and showed the huge improvement in response time. The Clinician would normally sit waiting for many seconds before anything would happen (the slowest community wins), but with the Async API they would get results right away (the fastest community wins). What surprised us both is that overall the complete results were available faster.

Alternative Patient Discovery performance improvement

The biggest advance is gained by the Patient Discovery. The alternative architecture is the one CommonWell has implemented as their Record Localization Service. They get the equivalent of an Asynchronous performance by requiring all Partners to feed them an ADT, for which they do cross-reference correlation at that time, so that when a Patient Discovery is needed the answer is already known fully.  This is what IHE has in the XDS environment. The negative is that all Patient Demographics are stored locally regardless of if that patient will ever utilize the network (some patients never leave home). The performance improvement is significant. Likely also much better quality of the matching is possible.

The websequencediagrams

+-------------------------+
title High-Level Community Exchange

participant Consumer
participant Service

Consumer->+Service: Patient Discovery (PD)
note right of Consumer
time happens (PD)
end note
Service->-Consumer: 10 Patients found

loop for each Patient at 10 Communities
Consumer->+Service: Query for Documents (QD)
note right of Consumer
time happens (QD)
end note
Service->-Consumer: 10 Documents found at that Partner Community
end loop

loop for each Document (10*10)
Consumer->+Service: Retrieve Document (RD)
note right of Consumer
time happens (RD)
end note
Service->-Consumer: returned document 
end loop


+-------------------------+
title High-Level Async Community Exchange

participant Consumer
participant Service

Consumer->+Service: Async Patient Discovery (PD)
Service-->Consumer: first Patient found
activate Consumer
par for each Patient 
Consumer->+Service: Async Query for Documents (QD)
deactivate Consumer

Service-->Consumer: first Document Entry at that Community
activate Consumer

par for each Document Entry
Consumer->+Service: Retrieve Document (RD)
deactivate Consumer
Service->-Consumer: returned document 
end par
Service->-Consumer: ... 10 Documents found at that Partner Community
note right of Consumer
each would kick off a par like above
end note 
end par
Service->-Consumer: ... 10 Patients found in network
note right of Consumer
each would kick off a par like above
end note 

+-----------------------------+
title Automate fully Async Community Exchange

participant App
participant Middleware
participant Service

App->+Middleware: Subscription (DocumentReference?Patient:fname="bob")
Middleware->+Service: Async Patient Discovery (PD)
note right of App
Middleware will notify App for each DocumentEntry found
end note
Service-->Middleware: first Patient found

activate Middleware
par for each Patient 
Middleware->+Service: Async Query for Documents (QD)
deactivate Middleware

Service-->Middleware: first Document Entry at that Community
activate Middleware

par for each Document Entry
Middleware->+Service: Retrieve Document (RD)
deactivate Middleware
Service->-Middleware: returned document 
end par
Service->-Middleware: ... 10 Documents found at that Partner Community
note right of Middleware
each would kick off a par like above
end note 
end par
Service->-Middleware: ... 10 Patients found in network
note right of Middleware
each would kick off a par like above
end note 
Middleware-->App: in realtime - notifiy of each more Document Entry
App->Middleware: GET DocumentReference...
App->Middleware: GET document

App->Middleware: deactivate Subscripiton




Thursday, September 6, 2018

PDQm using FHIR Subscription as Async API to XCPD/XCA

In the last article I covered the basic flow of using PDQm and MHD as an API to an XCPD/XCA environment. This is the classic intention of the current IHE Profiles.

But, the reality of XCA communities are that

  1. You likely have hundreds of communities to probe
  2. Some of those communities will be fast, some will be slow
  3. Some communities will be down and never respond

The reality is that the time between a Consumer asking a PDQm discovery request, and getting ALL of the results from ALL of the communities, might be many minutes. Yes, many minutes. That long of a time between a RESTful GET (query) and the response is unusual. A typical http RESTful api toolkit likely doesn't wait that long.

One solution is to just make sure all your Consumer implementations wait a very long time before giving up and assuming their Responder/InitiatingGateway infrastructure has failed them. How long to wait? This might be a value populated somewhere, like the CapabilityStatement on the Responder, or in a Service directory.

But that is not friendly to the user. You might be able to get the Consumer software to wait many minutes, but there is likely a human waiting that long too. It is possible to stream each result back as they become available, to the Consumer. I am not sure what that looks like. To stream the results would also require the design of the Consumer software to be expecting streamed results. Meaning it doesn't wait for the whole http response before it starts looking at what it does have... Again, not a friendly solution.

Lastly the Responder/InitiatingGateway must have a blocking thread for this many minutes too, taking up precious resources on the middleware.

Some of these problems are http RESTful, but most of them are general problems that exist even in the classic XCPD/XCA world using SOAP. 

FHIR has a solution ready to be used... Subscription.... With Subscription the Consumer creates a Subscription resource with the subscription criteria equal to the PDQm query it might otherwise do.  The Consumer gets notified as data matching that Subscription becomes available.

The typical use-case for Subscription is to register a subscription for new data that might appear in the future as created by someone else. In the use-case I am describing here it is as an Asynchronous mechanism for a set of transactions (XCPD). In my case the creation of the Subscription is a trigger to start some backend XCPD discovery. Once all of that XCPD discovery is done, then the Subscription is useless. So the end of the Subscription is different than typical use-case for Subscription.

I worked with a developer who implemented this as an internal API to their implementation that interfaces with the eHealth Exchange (Sequoia Network). Where there are well over 100 partners, and some partners take many minutes to respond. The result was a much more responsive application behavior, in that case a web-interface. In fact the whole response-time from start to complete finish was faster as the critical processing could happen during what would normally be wait time, rather than doing critical processing after all the network interactions happened. So double win!

This looks more busy, and it is.. but the waiting is between the gaps. The best part of this approach is that the "new results - ping 1234" will happen once for each Patient match found, and will happen when that match is available. Thus the Consumer software will learn of matches from fast partners really fast. The Consumer can then start doing something right then, like minimally displaying the results. Unlike the classic solution that prevents the Consumer from knowing anything until everything is known.

Note I used web-sockets here, which does require a the Consumer to query after each notification. There are other Subscription types that can pass the data back. I haven't experienced them, and don't see much written about them.

Here is the source for the UML diagram that can be used at https://websequencediagrams.com

+------------------------+
title PDQm Subscription -- XCA

participant Consumer
participant Responder
participant InitiatingGateway
participant RespondingGateway

note right of Consumer
based on some need a patient is of interest
end note over 
Consumer->+Responder: Create Subsubscription
note right of Consumer
Simply move the PDQm search on Patient 
into the Subscription criteria:
Subscription.criteria = Patient? blah
Subscription.channel = websockets
end note
Responder->Responder: check if valid request
Responder->InitiatingGateway: start discovery
activate InitiatingGateway
Responder->Consumer: Subscription Create success = 1234
Consumer->Responder: Open Web-Sockets channel
note right of Consumer
endpoint of Web-Sockets is identified 
in CapabilityStatement on Responder
uses websocket extension
end note
Consumer-->Responder: link this websocket to subscription -- bind 1234
Responder-->Consumer: successful link to subscription -- bound 1234

loop for --ALL-- Partners in Community
note right of InitiatingGateway
Design backend to be most responsive:
Could be thread pool, could be event driven 
Might use XCPD synchronous or asynchronous
end note
activate InitiatingGateway

InitiatingGateway->+RespondingGateway: XCPD Patient Discovery
RespondingGateway->-InitiatingGateway: found patient alpha @ beta community
InitiatingGateway-->Responder: store result in cache
activate Responder
deactivate InitiatingGateway
end loop

Responder-->Consumer: new results - ping 1234
deactivate Responder
activate Consumer
note right of Consumer
normal PDQm query
plus Consumer remembers the time it last queried
Find Patient?blah&_lastupdated=time
end note
Consumer->+Responder: PDQm Patient?blah&_lastupdated=time
Responder->-Consumer: results from Cache
note right of Responder
cached entries returned 
can be maintained or flushed
end note
deactivate Consumer
InitiatingGateway->Responder: notice all Partners have responded, or timeout
deactivate InitiatingGateway
note right of InitiatingGateway
Given no more can happen then done
end note 
Responder->Consumer: Close web-socket

Consumer->Responder: Update Subscription(1234) inactivae
deactivate Responder
note right of Consumer
at any time Consumer can inactivate
end note
note right of Responder
take down web-socket if still up
flush cache 
end note

Wednesday, September 5, 2018

MHD as an API to XCA

I expected this configuration was well enough explained within the MHD profile with the one paragraph and one diagram in the informative section 33.6.2-1.

I find that I need to explain this a bit more than I expected, and have a follow on article that needs this baseline.

Often I have to address the fact that XCA is a federation protocol that is addressing many communities. Federation is an important architectural capability allowing many communities to each act on-their-own, while cooperating in a Patient centric way.

The concept of simply using MHD as an API for XCA is over simplifying what actually needs to be done. Even the following diagram still oversimplifies in that not just PDQm and MHD are needed, but also ATNA for basic security, and IUA (or SMART-on-FHIR) paired with XUA for app and user security. This pairing of OAuth for FHIR with SAML for SOAP is not trivial, but is also not an unusual configuration for these security protocols. I am sure there is support in these product/stacks for this. Likely somewhere is needed a Consent management that might be based on BPPC or APPC.

What I will explain is the interactions between PDQm/MHD and XCPD/XCA.  The following diagram shows on the left the FHIR transactions, and on the right the SOAP transactions. This diagram shows the three steps: Patient discovery (PDQm-->XCPD), Document query (MHD-->XCA), and Document retrieve (MHD-->XCA). The three steps are joined by some processing that happens on the client side.


It is very possible that the processing on the client side is to simply do a Document query against all Patients found, and for each DocumentReference returned a Document retrieve. However I would hope that some kind of processing happens to select only the communities of interest, and only the documents of interest.

I don't show the design needed to convert PDQm (FHIR queries) into XCPD queries. I assert that for a given set of normally used queries this is possible. Note that the normally used queries are still a much smaller capability than PDQm or XCPD support. So, think about what is minimally needed, and work on that first.

I don't show the design needed to convert the MHD (FHIR queries) into the XCA queries. I assert that for a given set of normally used queries that are possible, there is a subset that is not hard to convert from FHIR to XCA. See Timebound XDS queries done right

Note that I have not done any extensions beyond what is documented in the PDQm/MHD or XCPD/XCA profiles. It is possible to do some short-cuts. It is possible to do some optimizations. I however here just wanted to put the facts on the table.

The above diagram is built from https://websequencediagrams.com using the following script. I provide it here for your use and enhancement, but also for my own archive of it:

+------------------------------+
title MHD -- XCA

participant Consumer
participant Responder
participant InitiatingGateway
participant RespondingGateway

note left of Responder
Consumer is used as PDQm-Consumer, and MHD-DocumentConsumer
Responder is used as PDQm-Supplier and MHD-DocumentResponder
end note 
note over Consumer
based on some need a patient is of interest
end note over 
Consumer->+Responder: PDQm Patient? blah
Responder->Responder: check if valid request
loop for all Partners in Community
Responder-->InitiatingGateway: trigger discovery

activate InitiatingGateway

InitiatingGateway->+RespondingGateway: XCPD Patient Discovery
RespondingGateway->-InitiatingGateway: found patient alpha @ beta community
InitiatingGateway-->Responder: store result in cache
deactivate InitiatingGateway
end loop
Responder->Consumer: Found all Patient reesources in cache
deactivate Responder

note over Consumer
based on Consumer processing of 
Patient resources returned
some or all Patients are interesting
end note

loop for all Patient entries found
Consumer->Responder: MHD DocumentReference?Patient=ABC&class=XYZ 
activate Responder
Responder->Responder: check if valid request
Responder-->InitiatingGateway: trigger 
activate InitiatingGateway
InitiatingGateway->+RespondingGateway: XCA Find Documents for given parameters
RespondingGateway->-InitiatingGateway: (n) Document metadata entries found
InitiatingGateway-->Responder: store results in cache

deactivate InitiatingGateway
end loop

Responder->Consumer: Found all document metadata entries in cache
deactivate Responder

note over  Consumer
based on Consumer processing of
DocumentReference resources returned
some or all Document Entries 
describe Documents of interest
end note
Consumer->Responder: Retrieve XYZ document
note right of Responder
URL found in DocumentReference is just retrieved (GET)
That URL might have been encoded by Responder to
have all the information necessary to do the XCA 
or it might rely on information in cache
end note
activate Responder
Responder->Responder: check if valid request (look in cache)
Responder-->InitiatingGateway: trigger Retrieve
activate InitiatingGateway
InitiatingGateway->+RespondingGateway: XCA Retrieve Documents
RespondingGateway->-InitiatingGateway: document returned
InitiatingGateway-->Responder: trigger doc available
deactivate InitiatingGateway
Responder->Consumer: document returned
deactivate Responder