# Shale

Shale is a Ruby object mapper and serializer for JSON, YAML, TOML, CSV and XML.

It allows you to parse JSON, YAML, TOML, CSV and XML data and convert it into Ruby data structures, as well as serialize data structures into JSON, YAML, TOML, CSV or XML.

# Introduction

Working with data serialization formats directly can be painfull. This is especially true for XML. Let's consider this simple example of adding an address to a person using Nokogiri:

That's a lot of code for very simple use case. Anything more complex and code complexity increases exponentially leading to a maintanace problems and errors.

With Shale you can use Ruby objects to work with data converting it to/from JSON, YAML, TOML, CSV or XML.

Let's convert the same example to Shale:

That's much simpler and it stays simple when the code complexity increases.

# Prerequisites

• Ruby 2.6+

If you want to work with XML you will need one of:

• REXML
• Nokogiri
• Ox

If you want to work with TOML you will need one of:

• Tomlib
• toml-rb

Shale doesn't have external dependencies. It uses standard library's JSON, YAML and CSV parsers by default.

If you need more customizations you can use custom libraries. Out of the box, Shale provides adapters for REXML, Nokogiri, Ox and toml-rb, but you can provide your own adapters - see how to.

# Features

• Convert JSON, YAML, TOML, CSV and XML to Ruby data model
• Convert Ruby data model to JSON, YAML, TOML, CSV and XML
• Generate JSON and XML Schema from Ruby models
• Compile JSON Schema into Ruby models
• Out of the box support for JSON, YAML, CSV, Tomlib, toml-rb, Nokogiri, REXML and Ox parsers
• Support for custom adapters

# Installation

Add this line to your application's Gemfile:

And then execute:

Or install it yourself as:

# Convert data to Ruby

JSON, YAML and CSV are supported by default, but to work with XML you need to install XML parser and setup adapter. For example to setup REXML as an adapter use:

If you want to work with TOML you will need to install Tomlib parser and setup it as an adapter:

Converting data to Ruby is as simple as defining model classes and calling from_<format> method on this class. e.g. Person.from_json(json_doc)

# Convert Ruby to data

To convert Ruby to data just define model class, initialize object and call to_<format> method on it. e.g. Person.new(name: 'John Doe').to_json

# Convert Collections

Shale allows converting collections for formats that support it (JSON, YAML and CSV).

To convert data to Ruby array:

To convert Ruby array to data:

# Custom mappings

When you define a class and add attributes, underneath Shale creates an implicit mapping of keys (for JSON/YAML/TOML), columns (for CSV) or elements (for XML) to attributes. That is nice for setting up your data model quickly, but usually your data format doesn't match your data model so cleanly.

That's why you can explicitly map keys, element and attributes from your data format to attributes on your Ruby model.

XML is more complex format.

• To map XML element use map_element
• To map XML attribute use map_attribute
• To map XML text node use map_content (it will map first text node of an element)
• To change the name of the root element use root

You can use cdata: true option on map_element and map_content to handle CDATA nodes:

# Using XML namespaces

To Work with XML namespaces you need to use adapter that supports XML namespaces. Use one of Shale::Adapter::REXML or Shale::Adapter::Nokogiri. Shale::Adapter::Ox doesn't support namespaces.

To map namespaced elements and attributes use namespace and prefix properties on map_element and map_attribute

To define default namespace for all elements use namespace declaration (this will define namespace only on elements, if you want to define namespace on an attribute, explicitly declare it on map_attribute).

# Rendering nil values

For JSON, YAML, TOML and XML by default elements with nil value are not rendered. You can change this behavior by using render_nil: true on a mapping.

For CSV, the default is to render nil elements.

If you want to change how nil values are rendered for all mappings you can use render_nil method:

# Using methods to extract and generate data

If you need full controll over extracting and generating data you can use methods to do so.

You can also pass a context object that will be available in extractor/generator methods:

If you want to work on multiple elements at a time you can group them with group block:

# Attribute delegation

To delegate fields to child complex types you can use receiver: :child declaration on a mapping.

# Additional options

You can control which attributes to render and parse by using only: [] and except: [] parameters.

By default generated JSON and XML are compacted. If you need human readable format use pretty: true parameter on #to_json and #to_xml

You can also add an XML declaration by passing declaration: true and encoding: true to #to_xml. You can use specific version by using string argument e.g. #to_xml(declaration: '1.1', encoding: 'ASCII')

For CSV you can pass headers: true to indicate that the first row contains column names and shouldn't be included in the returned collection. It also accepts all the options that CSV parser accepts.

# Using custom models

By default Shale combines mapper and model into one class. If you want to use your own classes as models you can do it by using model directive on the mapper:

# Generating JSON and XML Schema

To generate JSON or XML Schema from you Shale data model use:

You can also use a command line tool to do it:

or XML Schema:

If you want to convert your own types to Schema types use:

# Compiling JSON and XML Schema

To compile JSON or XML Schema and generate Ruby data model use Shale::Schema.from_json or Shale::Schema.from_xml. You can pass namespace_mapping: {} to map JSON schemas or XML namespaces to Ruby modules. For JSON schema, you can also pass root_name: 'Foobar' to change the name of the root type.

You can also use a command line tool to do it (JSON Schema):

and XML Schema:

# Supported types

Shale supports these types out of the box:

• Shale::Type::String
• Shale::Type::Integer
• Shale::Type::Float
• Shale::Type::Boolean
• Shale::Type::Date
• Shale::Type::Time

To add your own type define a class and extend it from Shale::Type::Value and implement .cast class method.

# Adapters

Shale uses adapters for parsing and generating documents. By default Ruby's standard JSON, YAML and CSV parsers are used for handling JSON, YAML and CSV documents.

You can change it by providing your own adapter. For JSON, TOML, YAML and CSV, adapter must implement .load and .dump class methods.

To handle TOML documents you have to explicitly set TOML adapter. Out of the box Tomlib is supported. Shale also provides adapter for toml-rb parser:

To handle XML documents you have to explicitly set XML adapter. Shale provides adapters for most popular Ruby XML parsers: