File Format

In order to do its job, the searchinator needs a way to store information about inators and users. For convenience, the searchinator stores this information in a nested dictionary.

All application data will be represented as a single Python dictionary. To make it easy to save this dictionary to a file and load it later, we will be using Python’s JSON module.

In short, the json module allows us to convert our data dictionary to a string. By doing this, we can write the dictionary to a file and load it later.

An example of a complete, JSON-encoded data dictionary with two inators and three users is shown here:

{
  "inators": {
      "ca23d1c7-6ac4-4930-ace1-62cc3120e271": {
          "name": "outside-in-inator",
          "added": "2017-09-23T03:33:37",
          "condition": 1,
          "ident": "ca23d1c7-6ac4-4930-ace1-62cc3120e271",
          "location": "movie theater",
          "description": "Morbi euismod feugiat hendrerit."
      },
      "fdd14918-2767-4aa8-9847-72f5381753f9": {
          "name": "other-dimension-inator",
          "added": "2017-09-26T01:33:37",
          "condition": 3,
          "ident": "fdd14918-2767-4aa8-9847-72f5381753f9",
          "location": "barber shop",
          "description": "Mauris cursus feugiat lacinia."
      }
  },
  "users": {
      "kevin": {
          "password": "banana",
          "username": "kevin"
      },
      "norm": {
          "password": "iMissGug",
          "username": "norm"
      },
      "heinz": {
          "password": "doof",
          "username": "heinz"
      }
  }
}

Data Generation

You’ll probably find it handy to generate sample data for the sake of development. generate.py can do just that. Here’s the command we used to generate the sample above:

python3.5 generate.py --inators=2 --credentials heinz:doof norm:iMissGug kevin:banana

Just specify how many inator records you want and any credentials you want to add. You can send the data straight to a file, as well:

python3.5 generate.py --inators=2 --credentials heinz:doof > inator_data.json

The > is used to redirect output to a file. It’s really handy for saving output from a program.

If you need help, check the usage docs:

python3.5 generate.py --help

Data Dictionary

We’ll refer to our dictionary of information as a “data dictionary”. It has the following structure:

{
    "inators": {
        "09849a15-27ee-4f30-9daa-acd74465864a": { ... },
        "d7258269-84a3-49ab-88c4-16927a529047": { ... },
        ...
    },
    "users": {
        "heinz": { ... },
        "norm": { ... },
        ...
    }
}

That is, the data dictionary contains two keys: inators and users. As implied above, the values for those keys are dictionaries, too.

inators

The value for the inators key is a dictionary that maps inator IDs (UUIDs) to inator records. This allows us to easily look up inator records by UUID:

data['inators']['09849a15-27ee-4f30-9daa-acd74465864a']  # Returns an inator record

The format of an inator record is detailed in the Inator Record Format section.

users

The value for the users key is a dictionary that maps a username to a user credential record. We can easily look up a user’s credentials by their username:

data['users']['norm']

The format of a user credential record is detailed in the User Credential Record Format section.

Inator Record Format

An inator record contains information about a single inator. In the following subsections, we’ll discuss how an inator record is represented as a Python dictionary and how it is represented as a JSON-encdoded string.

… as a Python Dictionary

Represented as a Python dictionary, an inator record look like this:

{
  'added': datetime.datetime(2017, 9, 25, 23, 39, 10),
  'condition': <Condition.ACTUALLY_WORKS: 5>,
  'description': 'Turns things upside down.',
  'ident': 'e0338076-d923-44d5-a863-d65eb2e7b3ab',
  'location': 'the basement',
  'name': 'upside-down-inator'
}

Each inator record has the following six keys:

  • added (datetime.datetime): The date when the inator record was added to the data dictionary.
  • condition (condition.Condition): The condition of the inator represented as an instance of condition.Condition.
  • description (str): A string that describes the inator.
  • ident (str): A randomly generated UUID that has been converted from a uuid.UUID to a str.
  • location (str): A string that location of the inator.
  • name (str): The name of the inator.

… as a JSON-Encoded String

When encoded as JSON, an inator record looks like this:

{
  "added": "2017-09-25T23:39:10",
  "condition": 5,
  "description": "Turns things upside down.",
  "ident": "e0338076-d923-44d5-a863-d65eb2e7b3ab"
  "location": "the basement",
  "name": "upside-down-inator",
}

Each inator record contains six keys:

  • added (str): The date when the inator record was added to the data dictionary. Should be a str formatted as %Y-%m-%dT%H:%M:%S in Python’s strptime syntax.
  • condition (int): The condition of the inator represented as an int. This number corresponds to the conditions enumerated by condition.Condition.
  • description (str): A string that describes the inator.
  • ident (str): The inator’s UUID as a str.
  • location (str): A string that location of the inator.
  • name (str): The name of the inator.

Dumping to a JSON-encoded string

By default, Python’s json.dumps() function does not know how to handle the added field, since it’s a datetime.datetime. That means we’ll have to tell it how to encode those special types. Whenever we want to dump an inator record to a str, we can do it like this:

# Given an inator record as a dictionary
s = json.dumps(record, default=utils.from_datetime)

This tells json.dumps() to call utils.from_datetime() whenever it doesn’t know how to encode a value as JSON. If you look at the source for utils.from_datetime(), all it does is convert datetime.datetime`s to :class:`str.

Note that if you’re dumping an entire data dictionary to a string, you use the same approach:

# Given a complete data dictionary
s = json.dumps(data_dict, default=utils.from_datetime)

There’s no need to dump individual dictionaries. Just pass the whole data dictionary to json.dumps() whenever you want to convert it to a string.

Loading from a JSON-encoded string

When we load an inator record from a string, json.loads() will not know to load added as a datetime.datetime or condidtion as a condition.Condition unless we tell it how to. In order to tell json.loads() how to load those objects, we’ll need another helper function:

# Given an inator record as a string
d = json.loads(record, object_hook=utils.as_inator)

This tells json.loads() to call utils.as_inator() whenever it decodes a dict from the JSON string. This gives us a chance to modify the dict during the decoding process.

If you look at the implementation of utils.as_inator(), you’ll see that if a dict looks like an inator record, we’ll attempt to load the added and condition fields. Otherwise, we just return the dict unmodified.

Note that you can load the entire data dictionary as above:

# Given a complete data dictionary
d = json.loads(data_dict, object_hook=utils.as_inator)

There is no need to try to call json.loads for individual records.

User Credential Record Format

A user credential record contains authentication information about a single user. In the following subsections, we’ll discuss how an inator record is represented as a Python dictionary and how it is represented as a JSON-encdoded string.

These are a lot simpler than inator records, since all we store here are str.

… as a Python Dictionary

Below is an example of a user credential record:

{
    'username': 'heinz',
    'password': 'doof'
}

Each user credential record has two fields:

  • username (str): The user’s username
  • password (str): The user’s password

… as a JSON-Encoded String

Below is an example of a user credential record:

{
    "username": "heinz",
    "password": "doof"
}

Each user credential record has two fields:

  • username (str): The user’s username
  • password (str): The user’s password

Unlike inator records, Python’s JSON module should have no issue dumping/loading user credential records to/from a string. We don’t need any special default or object_hook parameters.

That being said, there’s no issue that passing object_hook or default to handle inator records. Using those helper functions will not affect the JSON module’s ability to correctly load user credential records.