Testing your ATNA Audit Log implementation

Updated August 2016 New test tools for ATNA Audit Log messages in Gazelle

I am asked often for examples of ATNA Audit Log messages. IHE has far better than simple examples of ATNA audit messages.

First this information is part of a fantastic set of tools and infrastructure that the Connectathon team puts together. Their announcement to those that have registered has fantastic information around ALL profiles and how to test, prepare for connectathon and make the most out of the experience. PLEASE start with this information as it is critical to set the framework and background.

IHE has created an ATNA test tools, both client and server. The test tool for testing an ATNA Secure Node, that would be sending ATNA audit log messages, will catch and validate the message. It has an interface that shows you the messages that you have sent.

Syslog Collector info page: http://ihewiki.wustl.edu/wiki/index.php/Syslog_Collector

If your product is an ATNA Audit Record Repository then you can use their test tool to send various audit log messages to you. It has a web interface where you pick the message that you want to receive. Easy as pie.
Syslog Sender info page: http://ihewiki.wustl.edu/wiki/index.php/Syslog_Sender

The syslog sender tool itself has a drop down list of audit messages it can send: http://gazelle-gold.wustl.edu/SyslogSender/Sender.jsf

The Syslog Collector currently has complete ATNA validations (using Schematron) for the following messages. Note this is just a list I copied today from the wiki, go there for more details. Double-NOTE that this even points you directly to where the specification is for that audit log message.

• User Authenticated (Login) audit message ITI-2 (110123/1101140) TF-2a 3.2.6
• User Authenticated (Logout) audit message ITI-2 (110123/1101140) TF-2a 3.2.6
• Patient Identity Source Actor Audit Msg (Adm/Reg/Upd) (ITI-8/110110) TF-2a 3.8.5.1.1
• PIX Manager or Document Registry Actor Audit Msg (Adm/Reg/Upt) (ITI-8/110110) TF-2a 3.8.5.1.2
• Patient Identity Source Actor Audit Msg (Merge) (ITI-8/110110) TF-2a 3.8.5.2.1
• PIX Manager or Document Registry Actor Audit Msg (Merge) (ITI-8/110110) TF-2a 3.8.5.2.2
• PIX Consumer Audit Msg (ITI-9/110112) TF-2a 3.9.5.1.1
• PIX Manager Audit Msg (ITI-9/110112) TF-2a 3.9.5.1.2
• PIX Manager Update Notification Audit Msg (ITI-10/110110) TF-2a 3.10.5.1.1
• PIX Consumer Update Notification Audit Msg (ITI-10/110110) TF-2a 3.10.5.1.2
• Registry Stored Query Document Registry Audit Msg (ITI-18/110112) TF-2a 3.18.5.1.1
• Registry Stored Query Document Consumer Audit Msg (ITI-18/110112) TF-2a 3.18.5.1.1
• PDQ Consumer Audit Msg (ITI-21/110112) TF-2a 3.21.5.1.1
• PDQ Source Audit Msg (ITI-21/110112) TF-2a 3.21.5.1.2
• Patient Demographics and Visit Query Consumer Audit Msg (ITI-22/110112) TF-2a 3.21.5.1.1
• Patient Demographics and Visit Query Source Audit Msg (ITI-22/110112) TF-2a 3.21.5.2.1
• Provide and Reg Doc Set-b Document Source audit Msg (ITI-41/110106) TF-2b 3.41.7.1.1
• Provide and Reg Doc Set-b Document Repository or Document Recipient audit Msg (ITI-41/110107) TF-2b 3.41.7.1.2
• Register Document Set-b Document Registry Audit Msg (ITI-42/110107) TF-2b 3.42.7.1.1
• Register Document Set-b Repository or Src/Rep Audit Msg (ITI-42/110106) TF-2b 3.42.7.1.1
• Retrieve Doc Set Doc Consumer Audit Msg (ITI-43/110107) TF-2b 3.43.6.1.1
• Retrieve Doc Set Doc Repository Audit Msg (ITI-43/110106) TF-2b 3.43.6.1.2
• PID V3 PID Source Audit Msg (ITI-44/110110) TF-2b 3.44.5.1.1
• PID V3 PIX Mgr / Doc Registry Audit Msg (ITI-44/110110) TF-2b 3.44.5.1.2
• PIX V3 Consumer Audit Msg (ITI-45/110112) TF-2b 3.45.5.1.1
• PIX V3 Manager Audit Msg (ITI-45/110112) TF-2b 3.45.5.1.2
• PIX V3 Manager Update Notification Audit Msg (ITI-46/110110) TF-2b 3.46.5.1.1
• PIX V3 Consumer Update Notification Audit Msg (ITI-46/110110) TF-2b 3.46.5.1.2
• PDQ V3 Source Audit Msg (ITI-47/110112) TF-2b 3.47.5.1.2PDQ V3 Consumer Audit Msg (ITI-47/110112) TF-2b 3.47.5.1.1
• Value Set Consumer Audit Msg (ITI-48/110107) ITI_Suppl_SVS_Rev2-1_TI_2010-08-10 3.48.6.1.1
• Value Set Repository Audit Msg (ITI-48/110106) ITI_Suppl_SVS_Rev2-1_TI_2010-08-10 3.48.6.1.2
• Multi-Patient Stored Query Document Consumer Audit Msg (ITI-51/110112) TF-2b 3.51.5.1.1
• Multi-Patient Stored Query Document Registry Audit Msg (ITI-51/110112) TF-2b 3.51.5.1.2
• Document Metadata Subscriber Audit Msg (ITI-52/110112) ITI_Suppl_DSUB_Rev1-1_TI_2011-08-19 3.52.6.1.1
• Document Metadata Notification Broker Audit Msg (ITI-52/110112) IHE_ITI_Suppl_DSUB_Rev1-1_TI_2011-08-19 3.52.6.1.2
• Document Metadata Notification Recipient Audit Msg (ITI-53/110107) ITI_Suppl_DSUB_Rev1-1_TI_2011-08-19 3.53.5.1.1
• Document Metadata Notification Broker Audit Msg (ITI-53/110106) ITI_Suppl_DSUB_Rev1-1_TI_2011-08-19 3.53.5.1.2
• Document Metadata Notification Recipient Audit Msg (ITI-54/110107) IHE_ITI_Suppl_DSUB_Rev1-1_TI_2011-08-19 3.54.5.1.1
• Document Metadata Notification Broker Audit Msg (ITI-54/110106) ITI_Suppl_DSUB_Rev1-1_TI_2011-08-19 3.54.5.1.2
• XCPD Initiating Gateway Audit Msg (ITI-55/110112) ITI_Suppl_XCPD_Rev2-3_TI_2011-08_19 3.55.5.1.1
• XCPD Responding Gateway Audit Msg (ITI-55/110112) ITI_Suppl_XCPD_Rev2-3_TI_2011-08_19 3.55.5.1.2
• XCPl Initiating Gateway Audit Msg (ITI-56/110112) ITI_Suppl_XCPD_Rev2-3_TI_2011-08_19 3.56.5.1.1
• XCPL Responding Gateway Audit Msg (ITI-56/110112) ITI_Suppl_XCPD_Rev2-3_TI_2011-08_19 3.56.5.1.2
• Update Document Administrator Audit Msg (ITI-57/110106) ITI_Suppl_XDS_Metadata_Update_Rev1-2_TI_2011-08-19 3.57.4.1.4.1.1
• Update Document Registry/Recipient Audit Msg (ITI-57/110107) ITI_Suppl_XDS_Metadata_Update_Rev1-2_TI_2011-08-19 3.57.4.1.4.1.2
• Provider Information Source Audit Msg (ITI-59/110106) ITI_Suppl_HPD_Rev1-2_TI_2011-08-19 3.59.5.1.
• Provider Information Directory Audit Msg (ITI-59/110107) IHE_ITI_Suppl_HPD_Rev1-2_TI_2011-08-19 3.59.5.1.2
• Value Set Consumer Audit Msg (ITI-60/110107) ITI_Suppl_SVS_Rev2-1_TI_2010-08-10 3.48.6.1.2
• Multiple Value Set Repository Audit Msg (ITI-60/110106) ITI_Suppl_SVS_Rev2-1_TI_2010-08-10 3.48.6.1.2
• On-Demand Document Source Audit Msg (ITI-61/110106) ITI_Suppl_On_Demand_Documents_Rev1-2_TI_2011-08-19 3.61.7.1.1
• On-Demand Document Registry Audit Msg (ITI-61/110107) ITI_Suppl_On_Demand_Documents_Rev1-2_TI_2011-08-19 3.61.7.1.2
• XCF Responding Gateway Audit Msg (ITI-63/110106) ITI_Suppl_XCF_Rev1-1_TI_2011-08-19 3.63.6.1, see also TF-2b 3.41.7.1.1
• XCF Responding Gateway Audit Msg (ITI-63/110112) ITI_Suppl_XCF_Rev1-1_TI_2011-08-19 3.63.6.1, see also TF-2a 3.18.5.1.1
• XCF Initiating Gateway Audit Msg (ITI-63/110112) ITI_Suppl_XCF_Rev1-1_TI_2011-08-19 3.63.6.1, see also TF-2a 3.18.5.1.1
• Notify XAD-PID Link Change PIX Manager Audit Msg ITI-64/110110 ITI_Suppl_XPID_Rev1-1_TI_2011-08_19 3.64.5.1.1
• Notify XAD-PID Link Change Document Registry Audit Msg ITI-64/110110 ITI_Suppl_XPID_Rev1-1_TI_2011-08_19 3.64.5.1.2

Identity Proofing and Authentication -- Patient vs Provider

The HIT Policy Privacy and Security Tiger team is going to be looking into Patient identity credentials
Misuse/Fraud ID Proofing (including re: attributes, in-person, online and delegated registration) Authentication (including re: attributes and online challenges, two-factor authentication, credentialing, third-party authentication) Usability (including workability of solutions, complexity for patients)

I am a member of the HIT Standards Privacy and Security workgroup, so I eagerly await this testimony  GE Healthcare has been invited and will speak to the experience with the Patient Portal that is part of our Centricity EHR product. I will leave this  testimony stand alone.

From my perspective I look at this with RISK in mind. I certainly hope that 2-factor authentication is not needed by patients. I, as a patient, would be very annoyed by that, and it is simply not justified. Healthcare Providers are different, and I could get behind a multi-factor effort there for specific workflows (use-cases).

The difference is that a patient only has access to their own data, thus a failure exposes only ONE individual. Where providers have access to a very large number of patient data, one might say ALL possible patients. Thus a failure on provider is high risk. The Risk profile for Healthcare Providers and others using an EHR are clearly higher, but so are the ability of their environment to sustain more complexity. Note that I am not saying that ONE individual exposure is acceptable, I am saying that the risk profile is simply different and thus should be assessed.

I would like to see a risk profile put together on Patient Identities, as well as Provider Identities. This would look at the use-cases of these identities and misuse-cases. By putting together realistic Threats one can do realistic Impact assessment and Likelihood assessments. The Impact value is the most likely to be different, as I stated above. Without understanding the use-cases, misuse-cases, and risks; I fear that fear is all that will  be used to justify very expensive solutions.

I want to make sure that whatever is presented as ‘current state’; and that the healthcare industry continue to pursue the NSTIC efforts currently underway (for which I am participating). I want healthcare to NOT do something special, thus non-standard.

UPDATE: The GE testimony to the HIT Standards committees is published.

User Identity and Authentication

• Level setting on Level of Assurance 
• Advanced Access Controls to support sensitive health topics – a simple solution to sensitive health. 
• Direct addresses- Trusted vs Trustable 
• Identity - - Proofing
• The Emperor has no clothes - De-Identification and User Provisioning
• What User Authentication to use?
• IHE - Privacy and Security Profiles - Enterprise User Authentication
• IHE - Privacy and Security Profiles - Cross-Enterprise User Assertion
• Healthcare use of Identity Federation
• Federated ID is not a universal ID
• Separation of Layers: Security Error Codes  
• Authentication and Level of Assurance

MU2 - Why must healthcare use custom software when Thunderbird and Outlook would do?

Now that we have the test procedure for Transmit, the die is cast to continue to force Healthcare to use customized software even though we are so close to using Off-The-Shelf secure e-mail such as Thunderbird and Outlook.

TE170.314.b.2 – 2.05: Using the Inspection Test Guide, the Tester shall verify that the EHR technology is able to correctly discover and use address-bound and domain-bound certificates hosted in both DNS and LDAP

This test script correctly indicates that the EHR technology under test must do both DNS and LDAP methods for certificate discovery. This is indeed the way that the 1.1 version of the Direct project specification says, that both methods must be tried. This is even the recommendation of the S&I Framework project that requested that LDAP be added to the Direct project specification. I have commented against this requirement at all levels, and will continue to comment against it. I didn't use my negative vote, as I recognized that I was but a single vote.

What the issue is, when you have 2 methods to do the same thing one must define how you will get Interoperability between the methods. The two methods are the DNS-CERT and LDAP. What I mean is that you either force all certificates to be dual-published in both methods, or you force all clients to query both methods. The specification choose to query both methods. This WILL work. I am not arguing that it won’t work. But it has consequences that are not fully recognized.

I really don’t understand why this model was chosen. I think the thinking was that senders should be as robust as possible and try both protocols. I don’t disagree with this robustness principle. What I disagree with is FORCING this principle, that is forcing that all senders must do both methods. Since you are forcing that both methods are tried, you are forcing Providers and Patients to use customized software for healthcare. If this was not forced, then off-the-shelf e-mail such as Thunderbird or Outlook could be used. But since these off-the-shelf emails don’t do the DNS-CERT certificate discovery methods they can’t be used, even though they do a fine job with the LDAP lookup and the S/MIME handling.

Those that are hosting HISP or Identity services should be the most capable of hosting 2 services rather than just one. Thus my recommendation was that certificates be published in both methods. The advantage of this is that it wouldn't have invalidated early Direct clients (which forcing both does), and would have allowed for off-the-shelf e-mail. Further advantage is that we could have retired the method that was not adopted as much over time. Meaning that we would have had a way to transition to the winning protocol, which ever it might be. I am sure LDAP will win, but that is not important to the point.

One last point. Forcing dual publication on the service side would have impacted fewer sites than the forcing of the dual method on the sending side. The reason is simple, there will be more senders then there will be service sides. Simple observation says that there will surely be organizations that will be publishing many number of certificates for endpoints within their organization (Surescripts, HealthVault, etc). This math alone makes it clear there will be fewer services then there will be senders.

LDAP will win in the end because it provides for Healthcare Provider Directories, and other directories as well. Directories enable pick-lists of endpoint addresses, queries, and other workflows. Note that I do agree that DNS-CERT is likely a short-term win as we haven't quite nailed down what a Healthcare Provider Directory is, how one would find them, and such. 

Slightly concerning is the test step

TE170.314.b.2 – 2.07: The Tester cause the EHR to register the Direct (To) addresses specified in the Transport Testing Tool to be available as a recipient for sending of Direct messages within the EHR

It is possible that the EHR technology might want to have a pick-list for destination addresses, but surely the EHR technology better be able to take just a typed e-mail address. If we really are going to constrain to a pick-list, then why do we need ‘universal discoverability . With a ‘pick-list’ why do we need to have all this discussion around Certificate Authority – and Identity Proofing. We would just need a directory of ‘approved’ addresses. Heck we wouldn't even need S/MIME, as we could have gotten by just fine with a walled-garden security model…

More:

• Stepping stone off of FAX to Secure-Email
• Patient Portal - view, download, TRANSMIT
• Direct addresses- Trusted vs Trustable
• A broadly usable HIE Directory
• Healthcare Provider Discoverability and building Trust
• Healthcare Provider Directories Profile

MU Patient Engagement - Activity History Log

MU2 certification criteria will provide EHR technology that empowers the patient to see, online, an "activity history log" of when they themselves used the View, Download, and Transmit function. Not much, but a start. As clear as this criteria is in the MU2 regulation, I find it interesting how it can be misunderstood. This is one of the places where meeting the minimal criteria can be quite limiting. The limitation is architecturally the right stepping stone but I hope, as a patient myself, there is interest in going beyond the minimal criteria. We all must recognize that going beyond the minimum is not trivial.

The “Activity History Log” is specific to patient engagement  "§ 170.314 (e) Patient engagement. (1) View, download, and transmit to 3rd party."

(ii) Activity history log. (A) When electronic health information is viewed, downloaded, or transmitted to a third-party using the capabilities included in paragraphs (e)(1)(i)(A) through (C) of this section, the following information must be recorded and made accessible to the patient:
(1) The action(s) (i.e., view, download, transmission) that occurred;
(2) The date and time each action occurred in accordance with the standard specified at §170.210(g); and
(3) The user who took the action.
(B) EHR technology presented for certification may demonstrate compliance with paragraph (e)(1)(ii)(A) of this section if it is also certified to the certification criterion adopted at § 170.314(d)(2) and the information required to be recorded in paragraph (e)(1)(ii)(A) is accessible by the patient.

ONC is requiring only that the patient be given access to the audit events they-themselves caused in their-own actions related to View/Download/Transfer. The preamble text does clarify that this Activity History Log would also include any accesses by ‘authorized representative', although never defining how that works. So the minimal 'activity history log' does not include the ‘accounting of disclosures’, nor the restricted view of accounting of disclosures that the EHR technology is aware of, nor the privacy advocate bailiwick the ‘all accesses to the EHR ‘ – aka Access Report….

The “Activity History Log” is not access to all of the general security audit log “§170.314(d)(3) Audit report(s)”. This was the focus of a preamble comment on page 80.

This certification criterion does not require an EP, EH, or CAH’s general EHR technology security audit log to be made available to patients online. However, the activity history log must be available online and readily accessible. We hope that the past two responses have helped clarify many scope-oriented points for these commenters because it was our proposal and our continued belief that the activity history log should be online and readily available for a patient (or their authorized representative) to review “on demand.”

This preamble comment makes a clear distinction that the patient is only gaining access to the Activity History Log, and that does not mean the same thing as access to the whole Security Audit Log. The Security Audit Log will include all accesses to all patient data by all users along with many other security relevant events. This log is a very sensitive record. It shows not only low grade patient information, but more so shows behaviors of the workflows within the organization that the patient has no right or reason to see.

This does not mean that the Activity History Log can’t be a ‘view’ or ‘report’ that is created from the Security Audit Log. Indeed this is a likely way to create the Activity History Log report that is made available to the patient.

The Preamble Page 79 says:

This aspect of the certification criterion was not intended to implement the Department’s proposal to give individuals a right to receive an “access report.” However, given this confusion, we have decided to change the paragraph heading for this part of the certification criterion to state “activity history log.” The purpose of this paragraph in the certification criterion is to simply require that EHR technology be able to monitor when a patient or their authorized representative(s) views, downloads, or transmits their health information to a third party. Those are the actions to which this paragraph referred in the proposed certification criterion. Put simply, this activity log is meant to assist a patient track the history of their actions or those of their authorized representatives.

How big does the log need to be?
Note the next paragraph is also important as it indicates that there is no minimal requirement for the age of the audit entries that must be supported. I suspect this statement has more to do with not setting expectations that are hard to achieve. The HIPAA Accounting of Disclosures had to have complex statements on this to work through the initiation phase vs long-term goal. I certainly expect that the Activity History Log should go back at-least a year, and the HIPAA Accounting of Disclosures goal of 7 years will likely become the de-facto standard as we approach 7 years of use. This effectively means no-one will start to purge their Activity History Log until 7 years of activity.

The time period for which the activity history log should be available is a policy determination that should be made by the organization who implements EHR technology certified to this certification criterion. Thus, we decline to specify a particular retention period in this certification criterion. What is necessary for certification is that an EHR technology can demonstrate that it can properly create such a log. As noted in our response directly above, we intend for “user” in this context to be the patient and any authorized representative(s) to whom they have provided access to view, download, and/or transmit their health information to a third party.

Conclusion:
The minimum requirements for the Activity History Log:

• Only the Patient Engagement actions by the patient or their authorized representative
• Only their View, Download, or Transmit 
• Must be made available online
• Format of the log is not defined more than: Who, What, When
• Time period of age of the log is undefined

As minimal as this is, going beyond it is not trivial. Some thoughts on what more could be done. I just don't want to have this imply that these additions are easy.

• Include the Accounting of Disclosures – not likely because the definition of “Accounting of Disclosures” is inclusive of many things not handled by the EHR technology, including many things that are fully manual. Clearly using ATNA, and applications that can report disclosures outside of the EHR could be used. See my discussion on getting the Accounting of Disclosures with ATNA
• Include the EHR mitigated Accounting of Disclosures – this is harder than it seems as most of the reasons an EHR would be used to access the healthcare information are for the purpose of treatment, payment, or hospital operations; or fall into the exceptions. Thus there are not that many audit log events that rise to the level of being an Accounting of Disclosures. Determining which audit events qualify often requires manual processing.
• Include the Access Report – technically possible, but likely uncomfortable to the healthcare provider business leadership. This concern includes employee privacy rights issues.
• The biggest problem with the Access Report is that patient data is accessed on many systems, not just the EHR. Thus one really needs a combined view across the whole organization. This is indeed what IHE-ATNA provides.
• Extend the time period to 7 years, as we approach 7 years of use.
• When the EHR is using more mature HIE technology, provide an activity history log of each time the patients’ health information is accessed through the HIE.

2014 Draft Test Methods: Wave Four Released for Public Review and Comment

Wave 4 added a few more Security functionalities. The short of it is that these functional requirements are rather broad, and thus the testing needs to be rather broad. I do however see the test procedure hitting upon the specific key functionalities without being overly specific to one architecture. There are exceptions, but I suspect they can be handled in testing. I am glad to see that the writers of the test scripts is leveraging the preamble to the rule, something that is really not a formal part of the regulation. For those that want to understand MU2 better, I think these test scripts are the place to go. They include the rule, the preamble text, interpretation, and test script.

I have comments below on the following test scripts:

• Test Procedure for §170.314(d)(1) Authentication, access control, and authorization
• Test Procedure for §170.314(d)(2) Auditable events and tamper-resistance
• Test Procedure for §170.314(d)(3) Audit report(s)
• Test Procedure for §170.314(d)(7) End-user device encryption
• Test Procedure for §170.314(e)(3) Secure messaging – ambulatory setting only

2014 Draft Test Methods: Wave Four Released for Public Review and Comment The Office of the National Coordinator for Health IT (ONC) is pleased to announce the release of the fourth set of 2014 Edition draft Test Methods (test procedures, tools, and applicable test data and files). All Test Methods will undergo public review and comment before being finalized and approved by ONC for use in testing and certification. The final set of Test Methods is expected to be available for use in early 2013. Comments and suggestions should be submitted to ONC.Certification@hhs.gov. All submissions should include "2014 Test Methods" in the subject line. Please be as specific as possible in your comment submission. For additional information, and to access the 2014 Edition draft Test Methods, please visit the 2014 Edition Draft Test Procedures webpage.

Test Procedure for §170.314(d)(1) Authentication, access control, and authorization
This test procedure does basic user provisioning, use of that user, and de-provisioning to prove that use is now forbidden. There is a good balance in the procedure on testing the key functionality.

The hard part will be an EHR that totally uses standards to do User Identity, user Authentication, and even user authorizations. Meaning that the EHR software is not the one that will be tested, but rather the third party identity management system. For example in large hospitals Microsoft Active Directory is commonly used. It is this software that will be primarily tested by this test procedure. Although the EHR will still need to prove that they act properly. So, in the end good testing will be done.

It was disappointing that they didn't leverage the steps in this test procedure to test that audit events were recorded. Back when I was involved with CCHIT, we had a test procedure that tried to optimize the testing of these kinds of security functionality. One workflow that would test many functions at the same time. Including audit logging here would have been an easy thing.

Test Procedure for §170.314(d)(2) Auditable events and tamper-resistance

A minor victory, the testers recognize that NTP is for time clock synchronization, not the format of a timestamp.

Failure: The tests still seem to think that the only way to protect an audit log is to have a hash across it. This is actually a very unlikely method, but legitimate for some architectures. The test in "Detect Audit Log Alteration" should really be functional, and not specific to the technology of using a hash.

Disappointing: I expect that NIST can come up with better and more correct ways to test NTP. The tests in "Network Time Protocol (NTP) Test" will not work if the time clocks are off by too big of a jump. Good NTP implementations will not skew the time when it is so far out of alignment, and with good reason. The NTP is a long-term maintaining protocol, not a protocol for gross adjustments. Please check within NIST, surely there are standalone tests already available for general use. Why invent a special test for healthcare, especially inventing and being wrong.

Audit testing of changes to patient data "DTR170.314.d.2 – 3: Record Actions": I really wish the tests verified that egregious health information didn't make it into the audit log. The audit log should be as free of health information as possible, yes it will be very full of identifiers. This is not because I think that the audit logs don't need to be protected, but rather as a risk reduction method. Given that there is no need for health information in the audit log, it shouldn't be there. Once it is there, then it raises the risk of inappropriate exposure. Recognize that audit logs are reviewed for many reasons by non-doctors.

Test Procedure for §170.314(d)(3) Audit report(s)
I think this test procedure is written well enough to be testing useful functionality without imposing a specific way to implement it. I would like them to spend less time focused on steps of "Generate Audit Report", then focus on "Sort Audit Log". These two functionalities can be just as effective when implemented as one step. That is to identify the filter criteria, sort criteria, and reporting layout - then execute to produce the report. I suspect that this architecture would be seen as compliant, it would just be rather clumsy getting it through the test.

Test Procedure for §170.314(d)(7) End-user device encryption
I spent much time in HIT Standards Privacy and Security workgroup meetings on this topic, so I hope that the result is truly useful. The first approach, which happens to be the second one listed, is to design your client software in a way that it simply doesn't leave behind PHI when it normally exits. There are all kinds of supporting comments in the preamble to recognize as out-of-scope misbehaving browsers, non-normal termination, etc. Note that the test steps don't recognize these exceptions, so make sure you remind them.

The criteria is mostly focused on EHR architecture that pull databases onto the client. Even these simply need to encrypt the database onto the client then they will be able to show they are compliant. Simple.

I do have to laugh that the test procedure asks the tester to prove that locally stored PHI is encrypted to FIPS spec... This should be impossible, right? The best the tester can do is take vendor claims and record them. Then inspect to see that it isn't obviously PHI. To the tester the data might simply be base64 encoded.

I would highly encourage ALL architectures to 'not get in the way of transparent and automated hard-drive encryption'. The real concern is that some EHR vendors are forbidding their customers to do this. I don't know how real this concern is. I find it hard to believe it really happens in EHR space (I know it happens in Medical Device space, but that is a very different issue).

Test Procedure for §170.314(e)(3) Secure messaging – ambulatory setting only
This test procedure is again well written. I am somewhat concerned that it is written a bit too close to a Patient-Portal functionality. This is likely the prevailing method that will be used. My concern is that it seems to be overly prescriptive on how the patient will interact with technology. Meaning if I am a patient that is using a tool like Keith is building for ABBI, I might have to do things that are not in the control of the user. Or if I am a patient using a standalone PHR, such as HealthVault, I would not be logging into the EHR but rather HealthVault. These other models are clearly outlined in the Preamble text, but it is simply not clear how they will be tested properly.

See also:

MU2 Wave 1 of Draft Test Procedures -- Integrity Problem

MU2 - Encryption and Hashing

The MU2 requirement gets specific about Encryption and Hashing.

§170.210(f) Encryption and hashing of electronic health information. Any encryption and hashing algorithm identified by the National Institute of Standards and Technology (NIST) as an approved security function in Annex A of the FIPS Publication 140-2 (incorporated by reference in § 170.299).

This Encryption and Hashing requirement is important but not hard to meet. The important part is that proprietary encryption is unacceptable, old encryption algorithms are unacceptable. Modern encryption (AES and SHA) are acceptable. The use of FIPS Publication 140-2 allows HHS and CMS to benefit from the intelligence community assessment of cryptographic algorithms, thus moving up automatically when the intelligence community does. The use of Annex A rather than the core FIPS 140-2 specification allows for relaxed rules around certification, this doesn't change the technical aspect but it does greatly reduce the invasive code inspection requirements of actual FIPS certification. The Annex A is very short, 6 pages long.

The summary:

• Encryption AES or 3DES; 
• Hashing SHA1, or higher; 
• Authentication is totally missing in MU2, although RSA is included in FIPS. 

Most of the Transports include fully security as part of the specification, so they are by definition already compliant with the Encryption and Hashing requirements.

• Direct – S/MIME authenticated using X.509 Certificates/Private Keys, Encrypted with AES128 or AES256, and Hashed with SHA1 or SHA256.
• Secure SOAP –secured with Mutual-Authenticated-TLS using X.509 Certificates/Private Keys, Encrypted with AES, and hashed with SHA1 (HMAC-SHA1). This specification also indicates that the user identity be provided using SAML assertions -- This is the same requirements that IHE has for XDR, XDS, XCA, etc...  through IHE-ATNA and IHE-XUA profiles.
• Secure HL7 v2 – There is no mention in MU2 of this dirty little secret, but all of those HL7 v2 requirements in the regulation would also need to meet the Encryption and Hashing requirement. The solution here is to use the Mutual-Authenticated-TLS as is used in the Secure SOAP stack. Many toolkits support this, but not all of them. At IHE Connectathon we run into people who have forgotten to test this, they usually get going quickly. As indicated above the low bar is just 3DES and SHA1; IHE starts with AES and SHA1.
• Patient Engagement - Secure Messaging – There is no guidance on what Secure Messaging is, and I think this is the right solution. But whatever is used for Secure Messaging must also meet the § 170.210(f) requirements. Given that the requirements are just focused on Encryption and Hashing; this is easily met with a typical HTTPS web-portal. This will only be server authentication, but the user would be authenticated and thus it is up to the user to use a trustable machine. Typical HTTPS will also include encryption and hashing of sufficient strength -- 3DES and SHA1 
• Data at Rest – End-user device encryption. -- Okay this isn't a transport, but whatever solution used to protect data at rest, it must also meet the Encryption and Hashing requirements. A good commercial solution or even the solutions built into operating systems cover this. What they don’t cover is KEY MANAGEMENT. If you don’t protect the key then it doesn't matter how well encrypted. Please leverage good solutions that were developed by specialists.
• Note that the best solution that is MU2 compliant is to NEVER save PHI onto the client workstation. This simply stops the risk at the root. This is not always easy, but is the best. 
• The Operational environment should be able to leverage transparent hard-drive encryption. 

More details on advanced topics

• Secure SOAP – End-to-End security - This is not in MU2 – In IHE there is an option to use WS-Security end-to-end security, but this requires also an update of common SOAP stacks and is administratively harder to achieve. Risk Assessment needs to drive the cost benefit. This model also suffers from the same problem Direct has in that to send something you must discover the endpoints Digital Certificate before you start the conversation. Where as TLS does the discovery inline as part of the protocol negotiation.
• Moving to SHA256 with TLS requires an upgrade. 
• Direct addresses- Trusted vs Trustable
• Identity - - Proofing
• Securing RESTful services
• Healthcare use of X.509 and PKI is trust worthy when managed
• SSL is not broken, Browser based PKI is
• Meaningful Use Stage 2 :: SHA-1 vs SHA-2
• Trusting e-Mail
• S/MIME vs TLS -- Two great solutions for different architectures
• Healthcare Provider Discoverability and building Trust
• Using both Document Encryption and Document Signature
• Document Encryption

Patient Portal - view, download, TRANSMIT

Transmit only, not receive. Much simplifying the EHR functionality and perfectly logical for patient and provider. This is NOT what I complained about in Meaningful Transmission into Oblivion. Totally different topic, and they fixed that.

I have been in quite a few discussions around very simply the "Patient" experience of using an EHR based Portal supporting View, Download, and Transmit. The questions are never around what it means for a patient to be able to View, that is rather obvious. It is not easy to pull off, likely one of the hardest parts to make work well. Download is even rather easy, it is some form of ABBI. Following Keith's recommendations there. Ill let Keith argue over the format that download should take, or even the technology. Ultimately MU2 does indicate the download is of the ambulatory summary or inpatient summary. I would like to have choice of the format, as the CCDA format isn't readable by me. I want it, but I also would like a nicely formatted PDF.

The concern that comes to me is what does it mean to give the patient the ability to "Transmit"? This question comes to me mostly because I have parsed the Transports specification with excruciating detail. I was also part of the Direct Project that created Transport (a), as well as very active member of almost all the other standards in (b) and (c).

Direct and only Direct
First there is the problem with the Transmit criteria has bound both the specification of what content must be transmitted with the specification of the transport. Further the transport is defined as exactly ONE transport [§170.202(a).]. They ONLY specify the (a) Transport. The Patient is forced to use the Direct Project as the transmit, at least by certification criteria. As we learned before that in practice the Direct Project is not forced. The Direct Project is only forced as a minimal Transport.

§ 170.210(e) Patient engagement. (1) View, download, and transmit to 3rd party. (i) EHR technology must provide patients (and their authorized representatives) with an online means to view, download, and transmit to a 3rd party the data specified below. Access to these capabilities must be through a secure channel that ensures all content is encrypted and integrity-protected in accordance with the standard for encryption and hashing algorithms specified at § 170.210(f).

...

(C) Transmit to third party. (1) Electronically transmit the ambulatory summary or inpatient summary (as applicable to the EHR technology setting for which certification is requested) created in paragraph (e)(1)(i)(B)(1) of this section in accordance with the standard specified in §170.202(a).
(2) Inpatient setting only. Electronically transmit transition of care/referral summaries (as a result of a transition of care/referral) selected by the patient (or their authorized representative) in accordance with the standard specified in §170.202(a).

Malicious Documents?
There is no requirement to UPLOAD or even RECEIVE. This is the first simplification from a technology perspective. Uploading brings along many issues that are amazingly numerous and much harder to deal with. For example by supporting upload or receive one must have mechanisms where a document from unknown provenance be introduced. This could at worse be a malicious document full of 'trojan' or 'worm' like code. The document might look like a perfectly good clinical document, but be falsified by the patient. At best the document might be from another provider and unchanged, but impossible to prove that it was unchanged. Far better to get the 'other' provider to provide their copy of the document through secure transports.

This from page 61-61 of the preamble comments

Comments. A commenter recommended that this certification criterion should include more specific capabilities than we proposed such as, accommodate patient generated data to “upload” into the EHR; include linkages to patient specific education materials; and be based upon a standing patient preference.
Response. We did not accept this recommendation. We believe the certification criterion is properly scoped to support its correlated MU objective and measure and do not seek to introduce additional burden that could be value-added functionality outside the scope of certification that EHR technology developers can include for competitive purposes.

Simplified Transport - Send from ONE identity.
The other positive about not including Receive is that it greatly simplifies the Transmit.  When a Patient is using a secure portal to interact with their healthcare provider EHR, they are a 'user' of that system. For the most part they are a very limited user, limited at least to only accessing their own records. As a limited user they are not on the same footing at an employee at the healthcare provider institution. As such, when the patient asks that their medical summary be Transmitted to someone else, the EHR doesn't necessarily need to send from the Patient identity. The first simplification is that the EHR can Transmit the document on behalf of the patient, that is, "by" a single identity such as 'the medical records department'. This is what would happen in the paper world, so there is precedent  In this way there is no need for the EHR to have individual identities for each Patient. All Patients can be represented by just one identity. Actually the Transmit function can always be done by 'the medical records department'. Thus the EHR only need have one identity, even for Provider delivery.

This does mean that the recipient of a Patient requested Transmit of a document can't respond back only to the patient. If they respond they will be responding back to the sending identity, likely 'the medical records department'. This might initially sound like a bad thing. Why would the original healthcare providing organization get 'between' the  patient and the recipient. The reasons are numerous and all good reasons. If the patient requested that  the  Transmit be to a Doctor; then that Doctor should have direct relationship with the patient. If the patient requested that the Transmit be with their PHR, then the patient can just use their PHR. Any Transmit back to the original sender should likely be intercepted, inspected, and carefully routed. This would be the responsibility of the medical records department.

Simplified Transport - No Receive at all -- No HISP
Further since there is no need to Receive, there really is no need to support inbound email for patients at the Healthcare Provider. The healthcare provider organization will likely need to receive something via Direct, but that might be a very different software module. I am focused on the Patient Portal as a standalone software module. That software module needs only support Transmit, it doesn't need to have an e-mail server exposed to the internet to receive e-mail. It doesn't need to publish certificates. It doesn't need to deal with all the more robust things one must do as a server vs a client.

Yes, this means that there is no need for a HISP. All the software functionality can work just fine behind the firewall of the  healthcare provider organization.

Send ONE document to ONE recipient
There is no stipulation in the regulation if one must be able to send more than one document at a time. I would recommend that if you support more than one document at a time, please use the XDM content package. It is designed to be simple yet help contain the content and explain the relationships of multiple content items.
There is also no stipulation that a patient be allowed to specify more than one recipient. The Direct Project tells you how to do this, but it really isn't needed. If the patient needs to send to more than one, then simply send multiple times.

Patient Portal Software Modules
Given the above, what does it take to be the "Transmit" portion if you are just a Patient Portal?

Administration:
A) You must get a Direct e-mail address for the sender. Likely this is simply configuration parameter. It comes from the Healthcare Provider organization, who likely just has one for their Medical Records Department -- MedRec@sunnyfamilypractice.org

B) You must get a Direct friendly Certificate issued for this e-mail address. This is the most difficult part because it is in the policy space that has not yet been settled. Note that you end up with a Private key that must be very carefully protected in addition to the Certificate which is expected to be public.

B.1) Normally you would need to publish this Certificate via both DNS and LDAP; but not really needed in this case. It might be published for other reasons, such as that it really is the Medical Records Department at the healthcare provider and they really do want to receive Direct messages for other reasons not associated with the Patient Portal support for Transmit.

C) You must be given the list of 'trustable' Certificate Authorities. These are all the 'authorities' that the Healthcare Provider organization will be trusting. This list likely comes from consensus including organizations like DirectTrust.org

D) You should be given a list of directories. This is part of the second method of certificate discovery that was added to the Direct specification. The standards used are tried and true, LDAP. Commonly used inside of organizations, in this case providing a "Healthcare Provider Directory" or "White Pages" on the internet. (see (c) below)

Software Components:

a) User Interface: Allows the user to select the document to send. Likely forcing only one be chosen.

a.1) In order to send more than one, it is best to use the XDM content packaging system.

b) User Interface: Allows the user to type in an e-mail address they want to send to. There really is no way to just look at the text and determine if it is a good address. yes it must meet e-mail address encoding, but that is not enough. So your software needs to try to find a valid certificate (see step (d))

c) User Interface: Could use the alternative certificate discovery mechanism from Direct to query the Healthcare Provider Directory -- White Pages. In this way assisting the Patient in finding the right e-mail address. Much creativity can be applied to this user interface.

d) Once you have a destination e-mail address you must determine if it is valid. This is done by finding a digital certificate via one and/or two methods identified in the Direct specification

d.1) DNS query: Given the e-mail address one just asks the internet DNS system if anyone knows of a certificate for the given e-mail address. You get back zero or many.

d.2) LDAP query: Given the list of directories given in (D) above, you query each looking for a certificate for the given e-mail address. You get back zero or many.

e) If you have one or more certificates then you need to look at each one of them and determine if they are well formed, correct for the given e-mail address, not expired, and the signature on the certificate validates to one of the trusted certificate authorities you were given in (C) above.

f) If you have zero certificates, or none that are good, then you clearly need to tell the patient that the e-mail address they gave is not correct. And start over.

g) You then create an e-mail message according to the Direct specification. Use the given e-mail address for TO:, and the FROM is the sending e-mail address from (A).

h) You then S/MIME the message (making it secure). This uses the certificate you discovered in (e) to encrypt the message, and the private key from the certificate from (B) to sign the message. You will also include the (B) certificate in the message for good housekeeping.

i) You then need to SMTP send the message. This has two parts, both rather easy. This is the part that all e-mailers need to do, so it should be readily available as a reusable software component.

i.1) DNS MX record lookup. This is a use of the DNS to ask the internet for the correct 'next hop', the next e-mail server, to send e-mail to with the given e-mail address. It typically returns the address of the mail server of the recipient. It can be some intermediary. Don't worry, as the message is well protected.

i.2) SMTP send. Your software will connect to the address that the DNS MX record said to connect to and using the SMTP protocol transfer the message to the target.

j) Please record in an audit log that this happened...

That is all there is. In detail. Just a few software components: smtp sender, DNS requester, Certificate validation, Certificate Authority store, Private key store, LDAP requester, and user interface. Most of the hard work is done on the Receiving side of Direct. For more see the Deployment Models on the Direct Project wiki

Updated:
Keith went and implemented all of this using the Direct Project reference implementation.Direct Transmission in Java : The Go-Cart Version