[Magento] How export and import are implemented?

Introduction

This long anticipated feature is arguably one of themost importantones in ecommerce world. Flexibility of this interface definescapability of the application to communicate with the outsideworld, be it legacy systems, 3rd party applications, priceaggregators, external order sources, fulfillment systems,accounting, etc.

Thus we took it very seriously and, as with everything elsein Magento wentone or more steps further than most systems. Effectivelyany import orexport is a conversion.From one medium to another. From database to local file,from FTP serverto database, from parsed HTML pageto Excel spreadsheet, from Google Spreadsheet to catalog.

Basically, it boils down to loading data from one resource,applying unlimited alterations/reformatting, and saving it toanother resource.

Welcome to the world of conversion.

Concept

Let’s take an example of exporting a product catalog to excelspreadsheet file. The logical steps are:

• Load records from database
• Convert internal identificators into human understandabletext
• Map database fields to our custom file headings (sku ⇒ PartNumber)
• Format data array as Excel   XML
• Save resulting text as local or remote file

This sequence of actions we call a Profile.

Here we have 4 categories of components:

• Adapter (input/output)
• Mapper (data value processing)
• Parser (data format conversion)
• Validator (data validation)

Adapters

Adapters are anything that can plug in to external resource. Thismay include:

• Local files
• FTP, SFTP servers
• Database table
• Soap services
• Standard input/output
• HTTP  interface
• HTML  source
• Google Spreadsheets
• Custom resources (specialized db interface, cache repository,etc.)

Adapters can have 2 methods:

• load  - extract datafrom resource into internal container.
• save  - save data intoresource.

Mappers

Mappers are for processing data without changing it’s format orstructure. They’re useful for tasks such as mapping one field toanother or apply formulas on values.

Mappers can run one method: map

Parsers

Parsers are for changing format of data. Examples:

• CSV text ⇒ Grid array
• Excel   XML  text⇒ Grid array
• Grid array ⇒ Product model collection

Parsers can have 2 methods:

• parse  - usually convertfrom more human readable data to machine friendly format
• unparse  - the opposite,into human readable format

You might notice that Parsers are separated from Adapters, whenusually they are associated as one thing (export to CSV file).

Validators

Validators are created to make sure that conditions are met. Theconditions can be applied to processed data or environmentvariables.

Validators can only validate.

Implementation

Let’s take an example of exporting catalog products into CSVfile.

1. <action type="catalog/convert_adapter_product" method="load">
2.       <var name="store">base </var>
3. </action>
4.  
5. <action type="catalog/convert_parser_product" method="unparse">
6. </action>
7.  
8. <action type="varien/convert_mapper_column" method="map">
9.       <var name="sku">Part# </var>
10. </action>
11.  
12. <action type="core/convert_parser_csv" method="unparse">
13.       <var name="delimiter">, </var>
14.       <var name="enclose"> <![CDATA["]]> </var>
15.       <var name="fieldnames">true </var>
16. </action>
17.  
18. <action type="core/convert_adapter_io" method="save">
19.       <var name="type">file </var>
20.       <var name="path"> </var>
21.       <var name="filename">products.csv </var>
22.       <var name="link">/dev/moshe/ magento/var/export/products.csv </var>
23. </action>