Shepherd-Herd#

Main Documentation: https://orgua.github.io/shepherd/tools/herd

Main Project: https://github.com/orgua/shepherd

Source Code: https://github.com/orgua/shepherd/tree/main/software/shepherd_herd

Shepherd-herd is the command line utility for controlling a group of shepherd observers remotely through an IP-based network. This is the key user interface for a private shepherd instance in the same network. The python package must be installed on the user’s local machine and sends commands to the sheep via ssh.

Installation#

shepherd-herd is a python package and available on PyPI. Use your python package manager to install it. For example, using pip:

pip3 install shepherd-herd

For install directly from GitHub-Sources (here dev-branch):

 pip install git+https://github.com/orgua/shepherd.git@dev#subdirectory=software/shepherd-herd -U

For install from local sources:

cd shepherd/software/shepherd-herd/
pip3 install . -U

Configuration#

All shepherd-herd commands require the list of hosts on which to perform the requested action.

Static Config#

To simplify usage you should set up an ansible style, YAML-formatted inventory file named herd.yml in either of these directories (highest priority first):

in your current working directory
relative to your current working directory in inventory/herd.yml
in your local home-directory ~/herd.yml
in the config path /etc/shepherd/herd.yml (recommendation)

Here is the example herd.yml-file in the inventory directory of the shepherd repository:

sheep:
  hosts:
    sheep0:
      ansible_host: 192.168.1.100
    sheep1:
      ansible_host: 192.168.1.101
    sheep2:
      ansible_host: 192.168.1.102
  vars:
    ansible_user: jane
# sheep can be assigned roles:
# - choices for time-sync: ntp_clients, gps_clients, ptp_clients, ptp_servers
# - to lock down the software there is a role "secured"
# - assign individually ("sheep0:") or as ranges ("sheep[2:10]:")
# https://docs.ansible.com/ansible/latest/user_guide/intro_inventory.html
# example:
ntp_clients:
  hosts:
    sheep0:
#    sheep[20:30]:

Note

Only the sheep:-block is needed by the tool.
IP-Addresses can be omitted if network is set up to resolve host-names.
To find active observers a ping-sweep (in this example from .1 to .64) can be achieved with:

nmap -sn 192.168.1.1-64

After setting up the inventory, use shepherd-herd to check if all your nodes are responding correctly:

shepherd-herd shell-cmd "echo 'hello'"

Or select individual sheep from the herd:

shepherd-herd --limit sheep0,sheep2, shell-cmd "echo 'hello'"

Dynamic Config#

This list of hosts is provided with the -i option, that takes either the path to a file or a comma-separated list of hosts (compare Ansible -i).

shepherd-herd -i sheep0,sheep1,sheep2, shell-cmd "echo 'hello'"

Command-Line Interface#

Note

The tool has integrated help-functionality. For a full list of supported commands and options, run shepherd-herd --help and for more detail for each command shepherd-herd [COMMAND] --help.

The command-line Interface is as follows:

shepherd-herd#

A primary set of options to configure how to interface the herd

shepherd-herd [OPTIONS] COMMAND [ARGS]...

Options

-i, --inventory <inventory>#: List of target hosts as comma-separated string or path to ansible-style yaml file

-l, --limit <limit>#: Comma-separated list of hosts to limit execution to

-u, --user <user>#: User name for login to nodes

-k, --key-filepath <key_filepath>#: Path to private ssh key file

-v, --verbose#

--version#: Prints version-infos (combinable with -v)

blink#

Helps to identify Observers by flashing LEDs near Targets (IO, EMU)

shepherd-herd blink [OPTIONS] [DURATION]

Arguments

DURATION#: Optional argument

distribute#

Uploads a file FILENAME to the remote observers, will be stored in REMOTE_PATH

shepherd-herd distribute [OPTIONS] FILENAME

Options

-r, --remote-path <remote_path>#: for safety only allowed: /var/shepherd/* or /etc/shepherd/*

-f, --force-overwrite#: Overwrite existing file

Arguments

FILENAME#: Required argument

emulate#

Use the previously recorded harvest-data (INPUT-PATH is a hdf5-file on the sheep-hosts) for emulating an energy environment for the attached sensor nodes and monitor their power consumption and GPIO events

shepherd-herd emulate [OPTIONS] INPUT_PATH

Options

-o, --output-path <output_path>#: Dir or file path for resulting hdf5 file with load recordings

-d, --duration <duration>#: Duration of recording in seconds

-f, --force-overwrite#: Overwrite existing file

-c, --use-cal-default#: Use default calibration values

--enable-io, --disable-io#: Switch the GPIO level converter to targets on/off

--io-port <io_port>#

Choose Target that gets connected to IO

Options:: A | B

--pwr-port <pwr_port>#

Choose (main)Target that gets connected to virtual Source / current-monitor

Options:: A | B

-x, --voltage-aux <voltage_aux>#: Set Voltage of auxiliary Power Source (second target)

-a, --virtual-source <virtual_source>#: Use the desired setting for the virtual source

-n, --no-start#: Start shepherd synchronized after uploading config

Arguments

INPUT_PATH#: Required argument

fix#

Reloads the shepherd-kernel-module on each sheep

shepherd-herd fix [OPTIONS]

harvest#

Simultaneously record IV data from the connected harvesting-sources on the chosen observers.

shepherd-herd harvest [OPTIONS]

Options

-o, --output-path <output_path>#: Dir or file path for resulting hdf5 file

-a, --virtual-harvester <virtual_harvester>#: Choose one of the predefined virtual harvesters

-d, --duration <duration>#: Duration of recording in seconds

-f, --force-overwrite#: Overwrite existing file

-c, --use-cal-default#: Use default calibration values

-n, --no-start#: Start shepherd synchronized after uploading config

inventorize#

Collects information about the observer-hosts -> saved to local file

shepherd-herd inventorize [OPTIONS] [OUTPUT_PATH]

Arguments

OUTPUT_PATH#: Optional argument

poweroff#

Power off shepherd observers. Be sure to have physical access to the hardware for manually starting them again.

shepherd-herd poweroff [OPTIONS]

Options

-r, --restart#: Reboot

program#

Programmer for Target-Controller

shepherd-herd program [OPTIONS] FIRMWARE_FILE

Options

-p, --target-port <target_port>#

Choose Target-Port of Cape for programming

Options:: A | B

-m, --mcu-port <mcu_port>#: Choose MCU on Target-Port (only valid for SBW & SWD)

-v, --voltage <voltage>#: Target supply voltage

-d, --datarate <datarate>#: Bit rate of Programmer (bit/s)

-t, --mcu-type <mcu_type>#

Target MCU

Options:: nrf52 | msp430

--simulate#: dry-run the programmer - no data gets written

Arguments

FIRMWARE_FILE#: Required argument

retrieve#

param ctx:: context
param filename:: remote file with absolute path or relative in ‘/var/shepherd/recordings/’
param outdir:: local path to put the files in ‘outdir/[node-name]/filename’
param timestamp:
param separate:
param delete:
param force_stop:

shepherd-herd retrieve [OPTIONS] FILENAME OUTDIR

Options

-t, --timestamp#: Add current timestamp to measurement file

-s, --separate#: Every remote node gets own subdirectory

-d, --delete#: Delete the file from the remote filesystem after retrieval

-f, --force-stop#: Stop the on-going harvest/emulation process before retrieving the data

Arguments

FILENAME#: Required argument

OUTDIR#: Required argument

run#

Runs a task or set of tasks with provided config/task file (YAML).

shepherd-herd run [OPTIONS] CONFIG

Options

-a, --attach#: Wait and receive output on shell

Arguments

CONFIG#: Required argument

shell-cmd#

Run COMMAND on the shell

shepherd-herd shell-cmd [OPTIONS] COMMAND

Options

-s, --sudo#: Run command with sudo

Arguments

COMMAND#: Required argument

start#

Start pre-configured shp-service (/etc/shepherd/config.yml, UNSYNCED when ‘time_start’ is not set)

shepherd-herd start [OPTIONS]

status#

Information about current state of shepherd measurement

shepherd-herd status [OPTIONS]

stop#

Stops any harvest/emulation or other processes blocking the sheep

shepherd-herd stop [OPTIONS]

Unittests#

For testing shepherd-herd there must be a valid herd.yml at one of the mentioned locations (look at Configuration) with accessible sheep-nodes (at least one).

Navigate your host-shell into the package-folder and
install dependencies
run the testbench (~ 30 tests):

cd shepherd/software/shepherd-herd
pip3 install ./[tests]
pytest