In the United States, kindergarten through grade 12 (K-12) schools are under increasing pressure to improve student performance, and to provide a more integrated view of student development. The Department of Justice, Department of Health, and other stakeholders involved in the success of students expect school districts and state departments of education to share information. Now, we consider a holistic view imperative for the development of tomorrow's workforce and citizens.
Achieving the goals for data movement and integration involves two key ingredients:
- Significantly greater movement of data from school districts to the state level for better consolidation, analysis, and recommendations.
- Greater integration across multiple stakeholders in a secure, usable fashion.
Until recently, most data integration requirements for K-12 schools concentrated on sharing select data within a school district, and select data movement with the state. A hub and spoke architectural pattern, popularized through the Schools Interoperability Framework (SIF), addressed the requirements fairly well. Characteristics such as guaranteed delivery and security were built in. Figure 1, from the SIF Association, shows the basic architectural pattern.
Figure 1. Schools Interoperability Framework
The core use pattern for SIF involves registration of interest by certain applications (through their SIF agent) for specific pieces of data, and registration by other applications of their capability to provide data, in a classic publication and subscription (often called pub/sub) message exchange pattern. Other capabilities include request/response message exchange and, in forthcoming releases, increased capabilities for the SIF agents. The core pattern and dominant use will remain with the pub/sub, data driven architectural paradigm.
To meet requirements for more movement of data from districts to the state level, and for greater integration between stakeholders, larger school districts and state departments of education are becoming more interested in service-oriented architecture (SOA). SOA involves integrating and choreographing distributed applications in support of business processes, including inter-agency processes. SOA fits well with the new business requirements facing K-12 school districts and state departments of education.
This article describes an interoperability bridge that allows state departments of education that are SOA based to interconnect and get data from districts that are SIF based.
The first requirement is to move significantly more data from the school districts to the state level. Movement of such data can be achieved through several protocols including FTP, web upload, SIF request/response, and web services.
The second requirement is that the data, once received by the state, must be:
- Consolidated into a common database
- Available for access by school districts and other stakeholders, such as higher education institutions, or the U.S. Department of Justice
The best method for making the data available from the state is through web services, which can provide ease of access and simplified integration. There is a profile of web services standards to provide security, reliability, and addressing using a robust integration mechanism.
A third requirement is that the data integrate with other applications within a state department of education. This requirement is easily achieved through SOA and web services.
As a state department of education establishes SOA as the integration architecture of choice, there is still a need to interact with school districts that have already invested in SIF. The need is especially relevant in the use case where a higher education institution requests transcripts for a student applicant. The request would typically flow through the state on to the appropriate school district (unless the state chooses to house every single student transcript). Hence, state applications must be able to work with a SIF-enabled district. A different use case involves data movement from school districts to the state, where the state is entirely SOA enabled.
There is now a SIF/SOA bridge that addresses the requirement that applications operating in the SIF paradigm can connect with applications in the SOA realm. The next section describes the interoperability bridge.
In the SIF architecture, agents front applications and act as the intermediaries for data communication between applications. Agents transform objects into SIF standard data objects, register themselves with the Zone Integration Server (ZIS), and tell the ZIS about their capabilities and interests. The ZIS routes requests and events appropriately between agents, guaranteeing message delivery and security. Essentially, in a SIF implementation, an application relies on its agents to exchange data using the ZIS.
To allow SOA applications to participate in data exchange with SIF zone applications, a SIF/SOA bridge was designed. It connects web services-based SOA applications to the SIF zone applications managed by the ZIS.
Figure 2 shows how a SIF/SOA bridge acts as a SIF agent in the SIF zone in order to integrate with a ZIS and the Data Objects in that zone. On the other side, the SIF/SOA bridge will expose a web service interface that enables an enterprise service bus (ESB), and other web service consumers, to connect to a SIF zone through web services.
Figure 2. Single SIF zone connected to SOA with SIF/SOA bridge
The following table outlines the main criteria for the design and implementation of the SIF/SOA bridge solution. It includes the functions that each component must support.
|Implement a SIF agent that conforms to the SIF specification that the bridge relies on to connect to the ZIS for exchanging data with SIF zone applications.|
|Implement and expose a web service to enable an ESB, and other web service consumers, to connect to a SIF zone through web services.||Data exchanges will flow from the SOA zones to the SIF zone. Allow
applications (web service consumer) in the SOA zone to: |
|Implement a web service client to call web service providers to enable data exchanges from the SIF zone to the SOA zones.||Allow call to web service provider in the SOA zone to:|
In addition to supporting the SIF agent and web service components, the bridge must also support converting and mapping message formats (both request and response) from a SIF zone application to the web service messages payload format, and vice versa.
The SIF agent, web service, and data transformation components are shown in Figure 3. You can see the interactions and flows among the components to carry out bidirectional data flow from systems in the SIF zone and the SOA zone. (View a larger version of Figure 3.)
Figure 3. SIF/SOA bridge design - key components and interactions
A simple scenario, called Student State ID validation, shows how the SIF/SOA bridge integrated the SIF zone local School Information System (SIS) running in a local school district and the state ID web service using the ZIS. In the scenario, the local SIS sends a student ID validation request to the state department of education identification system (State ID Service). Upon receiving the request, the State ID Service processes the request and sends the response back to the local SIS.
Figure 4 details the data flows and interactions among the components to carry out a request/response data exchange between the local SIS in the SIF zone and the State ID Service in the SOA zone.
Figure 4. SIF agent in SIF zone interacting with State ID web service through SIF/SOA bridge
In Figure 4, the components interact as follows.
- The district SIS system sends a
StudentLocatorobject to the ZIS.
- The ZIS sends back a SIF Acknowledgement to the district SIS system.
- The district ZIS sends this
SIFRequestto the SIF/SOA bridge's SIF agent.
- The SIF/SOA bridge's SIF agent extracts the SIF query (using the data transformation/mapping module) from the SIF Request and hands it off to the SIF/SOA bridge's web service component.
- The SIF/SOA bridge's SIF agent sends back a SIF Acknowledgement to the ZIS.
- The SIF/SOA bridge's web service component makes a web service call that forwards the SIF query to the State ID system's web service endpoint.
- Upon processing the State ID request, the State ID Service sends a
StudentLocatorobject, through a web services call, to the SIF/SOA bridge's web service component as a response to the original request.
- The SIF/SOA bridge's web services component extracts the SIF
DataObject(StudentLocator) and hands it off to the SIF/SOA bridge's SIF agent.
- The SIF/SOA bridge's SIF agent creates a SIF Response with the SIF
DataObject(StudentLocator) and sends the response to the originally requesting district's ZIS.
- The district ZIS sends a SIF Acknowledgement to the SIF/SOA bridge's SIF agent.
- The district ZIS forwards the SIF response to the local SIS system in response to the original SIF request.
- The district SIS system sends a SIF Acknowledgement to the district ZIS.
As mentioned, the SIF/SOA bridge provides the important capability of transforming web service messages (SOAP/HTTP) to SIF messages. It also manages connectivity with school districts' SIF zones. The following use case shows the real value of the SIF/SOA bridge: complex vertical data movement from local schools to the district level.
Imagine that an administrator at a higher education institution wants to request a high school transcript for a student who is in the application process with the institution. The administrator does not need to know how to connect and communicate directly with the local school districts. Instead, the administrator uses the single institution portal that connects to the appropriate state web service to request the transcripts. Unknown to the higher education institution, the state has two school districts that communicate with their integration platform in completely different ways: one school district only supports the SIF architecture, and the other supports only SOA.
Before diving into the different components needed to build a solution for this use case, let's outline the core requirements.
- The higher education institution must be able to send a transcript request to the state department of education.
- After processing the request, the state department of education must
return the appropriate transcript.
For the scenario, the transcript will be returned in the widely adopted Postsecondary Electronic Standards Council (PESC) standard format.
- The state department of education must be able to request the appropriate transcripts from two different districts: one that only supports the SIF architecture, and one that only supports SOA.
- For communication with the district that supports SIF architecture, a SIF-SOA bridge must be used. The district that supports SOA would only need standard web service support.
Figure 5 shows a high level architecture that could be used to meet the requirements of this use case.
Figure 5. Higher education institution requesting transcripts through state department of education
The architecture above includes an ESB at the state department of education to route and transform messages flowing between the higher education institution and the local school districts. The ESB contains several integration components that collectively enable the transcript request use case:
- Transcript request service to integrate with the higher education institution.
- Message communication with the SIF/SOA bridge to connect with the SIF district in standard web service protocols.
- Message routing from transcript request service to non-SIF district transcript application, or SIF district's SIF zone, depending on where a student is enrolled.
- Message transformation between SIF messages and PESC messages for SIF district to higher level institution communication.
- Message aggregation/de-aggregation to support the multiple SIF messages required to respond to a single higher education transcript request.
In one implementation of the use case, all of the integration components were implemented as mediation modules within WebSphere® Enterprise Service Bus. Instead of grouping all of the message routing, transformation, and aggregation within one mediation module, the mediation modules were designed to be extensible and flexible, resulting in several different mediation modules that had a specific purpose.
The sequence of data flows for the transcript request use case is shown in Figure 6.
Figure 6. Data flow for transcript request to school district using SIF/SOA interoperability bridge
In Figure 6, a transcript is being requested for a student that is enrolled in a SIF enabled district. You can follow the numbered flow and arrows (in red). Some highlights of the mediation components in this context include:
- The State Transcript Handler is a mediation module that decides whether the transcript request goes to the district that is SIF-enabled, or to the non-SIF district. The module routes the message to the SIF Transcript Handler or the PESC Transcript Handler, respectively.
- The SIF Transcript Handler is responsible for:
- Handling one input request message and response message.
- Sending two different SIF messages to the SIF district (using the SIF Bridge Message Gateway).
- Aggregating the two SIF message responses in order to build a PESC transcript.
- The SIF Bridge Message Gateway is responsible for all communication with the SIF district from the WebSphere Enterprise Service Bus through the interfaces provided by the SIF/WS bridge pipe.
Using an ESB, in addition to the SIF/SOA bridge, lets the state department of education provide a valuable, integrated service to the higher education institution that is seeking student transcripts. The higher education institution doesn't have to worry about the complexities of the SIF architecture.
Figure 7 shows four different patterns of deployment for the SIF/SOA bridge.
As outlined in Pattern 1, placing the SIF/SOA bridge at the state department of education to receive all SIF requests is one possible solution. In this pattern, the bridge must handle requests from multiple districts, or else have multiple instances of the bridge deployed at the state level (one per district).
Figure 7. Patterns of deployment for SIF/SOA interoperability bridge
In Pattern 2 above, a ZIS is deployed at the state department of education and is responsible for receiving all the SIF messages from the district applications. The SIF/SOA interoperability bridge connects with the ZIS, and translates all messages into web services messages for communication with other applications at the state. This approach avoids requiring a ZIS at every district.
Pattern 3 deploys the SIF/SOA interoperability bridge at each district, converting all communication from that district to the state into a web services communication. The state does not receive any SIF requests, and is purely web services based.
Pattern 4 is applicable where there is an intermediate, regional level data collection and forwarding point. The ZIS and SIF/SOA interoperability bridge can be deployed at the regional level such that all downstream two-way communication with districts is using SIF. All two-way upstream communication to the state uses web services.
Every deployment pattern has different benefits. Pattern use should be tailored to your specific requirements.
The IT staff at departments of education, school districts, and independent private schools face the challenge of doing more with less. IT budgets are being reduced, yet there's an expectation to increase the number of data reporting and business process services while improving their efficiency. As a result, IT departments no longer have the luxury of ripping and replacing existing services with new, proprietary, vendor-provided services. The rip and replace approach has an expensive up-front cost, and a downstream cost called vendor lock-in. With vendor lock-in it can be expensive to upgrade, and difficult to customize or integrate services around proprietary vendor technology.
Competent IT shops are expected to leverage existing services, and to incrementally add new services to the existing infrastructure. The new services are like Lego® blocks: easy-to-mange, independent components that snap together using standards. With this approach, services can be aligned with the business process. Because the services are linked together using standards, they can be easily swapped out, creating an environment where business processes are flexible and easy to extend.
There are many reasons why an organization will want to embrace SOA:
- There's a large marketplace for development tools, business process engines, and application servers. The tools are available from both open source communities and commercial vendors.
- Standards for linking the services together allow for a flexible infrastructure that's easier to support, enhance, and replace.
- Standards for the infrastructure middleware avoids vendor lock-in.
- SOA is a critical component of the cloud.
- SOA has proven itself in virtually every other industry. Why would education be different?
The SIF/SOA interoperability bridge is an important first step for an organization wanting to leverage existing investments in their SIF architecture. Agencies that are SOA based can interconnect and get data from applications that are SIF based. The bridge accommodates a standards-based SOA infrastructure that can grow over time.
- Schools Interoperability
Framework Association: Learn how the SIF Association brings
together the developers and vendors of school technology with the federal,
state, and local educators who use that technology.
- Service Oriented
Architecture: Explore SOA, a business-centric IT architectural
approach that supports integrating your business as linked, repeatable
business tasks, or services.
- "Adoption of SOA for
Enterprise Systems in Education: Recommended Practices," from the
IMS Global Learning
Consortium, filters the information on the current state of SOA
concepts, tools and practices and provides guidance on when adoption of
SOA is appropriate in Education to overcome some of its core challenges.
- IBM developerWorks
Industries: Get all the latest industry-specific technical
resources for developers.
- Industries library: See the developerWorks Industries library for
technical articles and tips, tutorials, standards, and IBM Redbooks.
- developerWorks Community: Personalize your developerWorks
technical events and webcasts: Stay current with technology in
- developerWorks on
Twitter: Join today to follow developerWorks tweets.
- developerWorks podcasts: Listen to interesting interviews and
discussions for software developers.
- developerWorks on-demand demos: Watch demos ranging from product
installation and setup for beginners to advanced functionality for
Get products and technologies
- Get the SIF
- IBM trial
software: Evaluate IBM software products in the method that suits
you best. From trial downloads to cloud-hosted products, developerWorks
features software especially for developers.
- developerWorks blogs: Check out these blogs and get involved.
Sri is an industry solution standards leader for the Education and Transportation Industries in IBM. He previously co-chaired the SOA Best Practices Technical Report at the IMS Global Learning Consortium.
Chris is an industry solution standards engineer in the Education, Transportation, and Chemical and petroleum industries. He has valuable development experience with various industry messaging standards, including the Schools Interoperability Framework, XML, XML Schema, and Java™ technologies.