A few weeks ago I wrote about the missing support for EDIFACT and derived standards in Open Source XML-based Transformation Tools such as XAware and Chainbuilder.
The goal was to get the people behind these communities motivated to provide the means and tools for managing Community projects. One of these will of course be the development of EDIFACT functionality.
Meanwhile the XAware Company has established a Sandbox environment for Community Members to start their development projects but moreover to enable knowledge transfer and share prebuilt transformation routines and connectors / adaptors.
The guidelines and procedures for contributing code to the core software are under development. These will include coding guidelines and standards, test and certification guidelines, contributor agreement required for developers wishing to contribute code to the project, acceptance criteria and procedures for the adoption into the core project.
One of the first Community projects will be the development of EDIFACT functionality. A short summary of the UN/EDIFACT message structure can be found in my bloart Short description of the EDIFACT Message Structure.
People from XAware-Europe, Atos Origin, freelancers and private persons are participating on a voluntary basis to establish the first XAware Sandbox project, the EDIFACT functionality. Last week the functional requirements and technical design were discussed in the Netherlands based on a draft proposal from one of the participants.
The goal is to establish a generic approach that will become available for the whole community. The preliminary functional requirements and technical design are described hereafter. You are welcome to join us in the development of the EDIFACT functionality or to comment on the preliminary requirements and design approach.
Background
XAware is Data Integration Middleware provided under the Open Software Foundation approved GNU General Public License Version 2 GPLv2. People all over the world are using XAware technology for various projects and purposes. The main reasons for the popularity are:
1. Benefits of Open Source software.
2. XAware has an open architecture and Application Programming Interface (API)
3. Solutions can be implemented in a short period of time
4. Reusable integration solutions for message standards
The benefits of Open Source software are:
- Lower software licensing costs
- Avoidance of proprietary lock-in
- Compliance with accepted industry standards
- High innovation speed
- Freedom to study how the programs works and adapt it to your needs
- Freedom to run the program
To use XAware as a generic transformation tool in large scale B2B environments the support for EDIFACT and alike standards is imperative.
EDIFACT contains a standard set of message definitions used for particular business functions such as shipment, warehousing and invoicing. This standard has been around for at least two decades and is expected to be replaced by a suitable XML standard in the future. Support for EDIFACT will allow XAware technology to be used for EDIFACT integration and EDIFACT migration projects where EDIFACT is replaced by an XML standard.
Purpose
This document describes the functionality to be provided and/or developed for EDIFACT support in XAware.
User requirements:
1. XAware must support reading, processing and writing of EDIFACT messages
2. Supported EDIFACT messages and specifics are:
- a. Message definitions from the UN/EDIFACT Standard Directories
b. Messages with zero or one Functional Group containing the contents of the message
c. XAware will not support customizing EDIFACT message structures such as adding or removing segments
. Subsets of EDIFACT messages
e. Parts of EDIFACT messages
3. EDIFACT support should be built under (commercial) open source conditions, meaning that it is reusable for all customers and users in the community
4. The Project, project documentation and other artifacts should be made available through the XAware community site and forum. This helps the community awareness of the EDIFACT initiative and reuse of the solution.
5. EDIFACT messages must be readable from multiple channels (file, queue, ws, etc)
6. EDIFACT communication configuration must be done at EDIFACT directory level
7. XAware must support multiple versions of the EDIFACT directory provided by UN/ECE
8. XAware must provide a solution that enables reuse of mappings between various standards
9. A methodology for future integration/migration projects using the EDIFACT solution should be defined. This methodology should address the following requirements:
- a. EDIFACT message discovery and exploration at business level
b. Mapping specification at business level
c. Mapping based on metadata (XML Schema Definition (XSD))
d. Reusable mapping components
e. Incremental process
f. Define the proper workflow in XAware designer to support this methodology
g. Select (or built in XAware) a powerful mapping facility (IDE) needed to support business consultants in defining the mapping from EDIFACT to other standards at the customer site
Functional requirements
1. XAware must read EDIFACT message definition from multiple files from the directory (edmd.zip, edsd.zip, eded.zip, edcd.zip, message definition)
2. XAware must implement logic to generate the corresponding EDIFACT XML XSD for a specified EDIFACT message definition.
3. XAware must map all messages to the corresponding ‘EDIFACT XML’ as the Common Information Model (CIM) and vice versa.
4. Reusable XAware mappings at segment level, composite data element and primitive data types
5. Separators for segments (‘’’), composite data elements (‘:’) and data elements (‘+’) should be made configurable. Reason is that the default values can be overridden in the message header.
6. The design of the solution should include a plug point for adding business logic (for validation or transformation)
7. The design of the segment processing logic should address
- a. Filter(s) for removing and selecting optional sections
b. Handle qualified data elements and data types (composite data elements & primitive data type conversion)
c. Reuse components that map segments, composite data types and primitive data types (e.g date time mappings).
d. Allow for generation of a skeleton BizDocument that implements the processing logic
8. Performance requirements are not clear at this time. Process messages up till certain size under a second.
9. Scalability requirements are not clear at this time.
Rough estimate: Technology should be able to cope with Millions of messages per day?
Technical Design
This section describes the design, considerations and decisions for implementing the EDIFACT functionality. The approach is to split development into two focus areas: the required XAware technology for working with the EDIFACT directories and the XAware scripts and process for building transformation routines.
XAware technology
1. Parser for reading EDIFACT message types and convert this to corresponding XML Schema Definition (XSD).
2. Instruction for processing messages in EDIFACT format similar to multi format instruction
- a. Parsing of the messages using EDIFACT directory (or EDIFACT metadata included in BizComponent at design time)
b. Convert the message to corresponding XML structure that conforms to the generated EDIFACT XSD and deliver that in the response
3. User interface wizard to create an EDIFACT BizComponent based on directory version and message id.
4. Where do we want to place EDIFACT directory in the architecture?
XA script
Building blocks for the segment processing logic are:
1. BizComponent that reads the EDIFACT message and returns the corresponding EDIFACT XML
2. (Generated) BizDocument skeleton that maps EDIFACT XML to the source/target system.
- a. Data type conversion logic?
b. Transformation logic?
c. Business logic?
3. XML mapper for handling repeatable groups in the EDIFACT XML
Last update: 26-11-2011