Bash Parse YAML: 4 Simple Examples

Introduction to Bash

Bash, or Bourne Again Shell, is a command interpreter and a well-admired Unix shell introduced by the GNU Project. Bash allows users to type text-based commands, which it interprets to perform a wide variety of tasks such as executing programs, manipulating files, and handling system operations. It’s popular due to its versatility and the powerful scripting environment it provides, allowing for automation of complex tasks.

Introduction to YAML

YAML, standing for “YAML Ain’t Markup Language”, is a human-friendly data serialization standard widely used in programming for data exchange, configuration files, and in applications where data is stored or transmitted. YAML focuses on simplicity and flexibility, which aids developers and system administrators in tasks like configuration management and data serialization. Further information can be obtained from the Official YAML documentation.

Why Parse YAML in Bash?

Parsing YAML in Bash is crucial as many software applications use YAML for configuration. It allows for easier and more streamlined manipulation and extraction of information from YAML files. Common scenarios where parsing YAML in Bash is needed include setting up server configurations, managing CI/CD pipelines, and developing applications with microservices architecture. In essence, Bash scripts parsing YAML data can automate complex and repetitive tasks, leading to improved productivity.

Parsing YAML in Bash Using Python and PyYaml

Python, coupled with the PyYaml library, offers a simple and effective way to parse YAML files in Bash. PyYaml provides a high-level interface for YAML, enabling Python scripts to effortlessly parse YAML. After setting up Python and installing PyYaml (pip install pyyaml), a Bash script can utilize Python to parse YAML files.

Example:

python -c 'import yaml,sys;print(yaml.safe_load(sys.stdin))' < example.yaml

Parsing YAML in Bash Using Ruby and yaml gem

Ruby with the yaml gem also offers a robust solution for parsing YAML in Bash. Ruby’s yaml gem can convert YAML files into Ruby objects, and vice versa. This gem can be installed with gem install yaml. A Bash script can then leverage Ruby to parse YAML.

Example:

ruby -ryaml -rjson -e 'puts JSON.pretty_generate(YAML.load(ARGF))' < example.yaml

Parsing YAML in Bash Using yq

Installing yq in Bash

yq can be installed in Bash using package managers like apt-get, brew, or by downloading the binary from the GitHub repository. The installation commands depend on the system you’re using. For Ubuntu, the command is sudo apt-get install -y yq.

Writing a Bash Script to Parse YAML Using yq

The Bash script for parsing YAML using yq consists of a command following the syntax: yq e '<expression>' <file>. <expression> contains the instructions for the parser, and <file>

Example:

#!/bin/bash

# Parse YAML file using yq
yq e '.example.key' example.yaml

Executing the Bash Script

A Bash script can be executed by giving the script execute permissions using chmod +x <script>, and then running it with ./<script>.

Example:

chmod +x parse.sh
./parse.sh

Parse YAML Array in Bash

Parsing a YAML array in Bash can be done with yq, a command-line YAML processor. Consider a YAML file example.yaml with an array of employee names:

employees:
  - John Doe
  - Jane Doe

You can parse this YAML array with a Bash script using yq:

#!/bin/bash
# Parse YAML array using yq
names=$(yq e '.employees[]' example.yaml)
echo "$names"

This script extracts each item in the employees array and prints the names “John Doe” and “Jane Doe” to the console.

Bonus: Converting Parsed YAML Data to SQL

The conversion of parsed YAML data to SQL involves transforming the data into a format that can be inserted into a SQL database. For instance, a parsed YAML file can be converted into SQL INSERT statements.

Example:

YAML file:

employee:
  - id: 1
    name: John Doe

SQL:

INSERT INTO employee (id, name) VALUES (1, "John Doe");

FAQ