Bash Parse YAML: 4 Simple Examples

Introduction to Bash and YAML

Bash, short for Bourne Again Shell, is a Unix-based command interpreter created by the GNU Project. It lets users enter text commands to run programs, manage files, and perform system tasks. Bash is widely used because it’s flexible and supports scripting, which makes automating complex tasks easier.

YAML, short for “YAML Ain’t Markup Language,” is a straightforward data serialization format. It’s commonly used in programming for data exchange, configuration files, and storing or transferring data. Its simplicity makes it a favorite among developers and system administrators for tasks like managing configurations and serializing data. Official YAML documentation offers more details.

Why Parse YAML in Bash?
Many software tools use YAML for configuration, making parsing it in Bash essential. This allows users to extract and work with YAML data efficiently. Common use cases include configuring servers, managing CI/CD pipelines, and building applications with microservices. Parsing YAML in Bash can automate repetitive tasks, saving time and boosting productivity.

Parsing YAML in Bash Using Python and PyYaml

Python, coupled with the PyYaml library, offers a simple and effective way to parse YAML files in Bash. PyYaml provides a high-level interface for YAML, enabling Python scripts to effortlessly parse YAML. After setting up Python and installing PyYaml (pip install pyyaml), a Bash script can utilize Python to parse YAML files.

Example:

python -c 'import yaml,sys;print(yaml.safe_load(sys.stdin))' < example.yaml

Parsing YAML in Bash Using Ruby and yaml gem

Ruby with the yaml gem also offers a robust solution for parsing YAML in Bash. Ruby’s yaml gem can convert YAML files into Ruby objects, and vice versa. This gem can be installed with gem install yaml. A Bash script can then leverage Ruby to parse YAML.

Example:

ruby -ryaml -rjson -e 'puts JSON.pretty_generate(YAML.load(ARGF))' < example.yaml

Parsing YAML in Bash Using yq

Installing yq in Bash

yq can be installed in Bash using package managers like apt-get, brew, or by downloading the binary from the GitHub repository. The installation commands depend on the system you’re using. For Ubuntu, the command is sudo apt-get install -y yq.

Writing a Bash Script to Parse YAML Using yq

The Bash script for parsing YAML using yq consists of a command following the syntax: yq e '<expression>' <file>. <expression> contains the instructions for the parser, and <file>

Example:

#!/bin/bash

# Parse YAML file using yq
yq e '.example.key' example.yaml

Executing the Bash Script

A Bash script can be executed by giving the script execute permissions using chmod +x <script>, and then running it with ./<script>.

Example:

chmod +x parse.sh
./parse.sh

Parse YAML Array in Bash

Parsing a YAML array in Bash can be done with yq, a command-line YAML processor. Consider a YAML file example.yaml with an array of employee names:

employees:
  - John Doe
  - Jane Doe

You can parse this YAML array with a Bash script using yq:

#!/bin/bash
# Parse YAML array using yq
names=$(yq e '.employees[]' example.yaml)
echo "$names"

This script extracts each item in the employees array and prints the names “John Doe” and “Jane Doe” to the console.

Bonus: Converting Parsed YAML Data to SQL

The conversion of parsed YAML data to SQL involves transforming the data into a format that can be inserted into a SQL database. For instance, a parsed YAML file can be converted into SQL INSERT statements.

Example:

YAML file:

employee:
  - id: 1
    name: John Doe

SQL:

INSERT INTO employee (id, name) VALUES (1, "John Doe");

FAQ