Python: Accessing YAML values using “dot notation”

Question

I\'m using a YAML configuration file. So this is the code to load my config in Python:

import os
import yaml
with open(\'./config.yml\') as file:
    config          


        

Answer 1

I generally follow a best practice of converting config (any kind, not just yaml) to an in memory object.

This way the text based config is unwrapped by 1 function and the text is thrown away, giving a beautiful object to work with as against having every function to deal with the internals of the config. That way all functions only know of that one internal object interface. If any new parameter is added/renamed/deleted from the config file, the only function to change is the loader function which loads the config into the in memory object.

Below is an example i did for loading FloydHub config yaml file into an in-memory object. I feel it is a very good design pattern.

First define a config representative class like below:

class FloydYamlConfig(object):
class Input:
    def __init__(self, destination, source):
        self.destination = destination
        self.source = source

def __init__(self, floyd_yaml_dict):
    self.machine = floyd_yaml_dict['machine']
    self.env = floyd_yaml_dict['env']
    self.description = floyd_yaml_dict['description']
    self.max_runtime = floyd_yaml_dict['max_runtime']
    self.command = floyd_yaml_dict['command']
    self.input = []
    for input_conf in floyd_yaml_dict['input']:
        input_obj = self.Input(destination=input_conf['destination'], source=input_conf['source'])
        self.input.append(input_obj)

def __str__(self):
    input_str = ''
    for input_obj in self.input:
        input_str += '\ndestination: {}\n source: {}'.format(input_obj.destination, input_obj.source)

    print_str = ('machine: {}\n'
                 'env: {}\n'
                 'input: {}\n'
                 'description: {}\n'
                 'max_runtime: {}\n'
                 'command: {}\n').format(
        self.machine, self.env, input_str, self.description, self.max_runtime, self.command)
    return print_str

Then load the yaml into the object for further use:

floyd_conf = read_floyd_yaml_config(args.floyd_yaml_path)

def read_floyd_yaml_config(floyd_yaml_path) -> FloydYamlConfig:
    with open(floyd_yaml_path) as f:
        yaml_conf_dict = yaml.safe_load(f)

    floyd_conf = FloydYamlConfig(yaml_conf_dict)
    # print(floyd_conf)
    return floyd_conf

Sample yaml

# see: https://docs.floydhub.com/floyd_config

machine: gpu2
env: tensorflow-1.0
input:
  - destination: data
    source: abc/datasets/my-data/6
  - destination: config
    source: abc/datasets/my-config/1
description: this is a test
max_runtime: 3600
command: >-
  echo 'hello world'