Shepherd-Herd

Main Documentation: https://orgua.github.io/shepherd/tools/herd

Main Project: https://github.com/orgua/shepherd

Source Code: https://github.com/orgua/shepherd/tree/main/software/shepherd_herd

Shepherd-herd is the command line utility for controlling a group of shepherd observers remotely through an IP-based network. This is the key user interface for a private shepherd instance in the same network. The python package must be installed on the user’s local machine and sends commands to the sheep via ssh.

Installation

shepherd-herd is a python package and available on PyPI. Use your python package manager to install it. For example, using pip:

pip3 install shepherd-herd

For install directly from GitHub-Sources (here dev-branch):

pip install git+https://github.com/orgua/shepherd.git@dev#subdirectory=software/shepherd-herd -U

For install from local sources:

cd shepherd/software/shepherd-herd/
pip3 install . -U

Configuration

All shepherd-herd commands require the list of hosts on which to perform the requested action.

Static Config

To simplify usage you should set up an ansible style, YAML-formatted inventory file named herd.yml in either of these directories (highest priority first):

in your current working directory
relative to your current working directory in inventory/herd.yml
in your local home-directory ~/herd.yml
in the config path /etc/shepherd/herd.yml (recommendation)

Here is the example herd.yml-file in the inventory directory of the shepherd repository:

sheep:
  hosts:
    sheep0:
      ansible_host: 192.168.1.100
    sheep1:
      ansible_host: 192.168.1.101
    sheep2:
      ansible_host: 192.168.1.102
  vars:
    ansible_user: jane
# sheep can be assigned roles:
# - choices for time-sync: ntp_clients, gps_clients, ptp_clients, ptp_servers
# - to lock down the software there is a role "secured"
# - assign individually ("sheep0:") or as ranges ("sheep[2:10]:")
# https://docs.ansible.com/ansible/latest/user_guide/intro_inventory.html
# example:
ntp_clients:
  hosts:
    sheep0:
#    sheep[20:30]:

Note

Only the sheep:-block is needed by the tool.
IP-Addresses can be omitted if network is set up to resolve host-names.
To find active observers a ping-sweep (in this example from .1 to .64) can be achieved with:

nmap -sn 192.168.1.1-64

After setting up the inventory, use shepherd-herd to check if all your nodes are responding correctly:

shepherd-herd shell-cmd "echo 'hello'"

Or select individual sheep from the herd:

shepherd-herd --limit sheep0,sheep2, shell-cmd "echo 'hello'"

Dynamic Config

This list of hosts is provided with the -i option, that takes either the path to a file or a comma-separated list of hosts (compare Ansible -i).

shepherd-herd -i sheep0,sheep1,sheep2, shell-cmd "echo 'hello'"

Command-Line Interface

Note

The tool has integrated help-functionality. For a full list of supported commands and options, run shepherd-herd --help and for more detail for each command shepherd-herd [COMMAND] --help.

The command-line Interface is as follows:

shepherd-herd

Entry for command line with settings to interface the herd.

shepherd-herd [OPTIONS] COMMAND [ARGS]...

Options

-i, --inventory <inventory>: List of target hosts as comma-separated string or path to ansible-style yaml file

-l, --limit <limit>: Comma-separated list of hosts to limit execution to

-u, --user <user>: User name for login to nodes

-k, --key-filepath <key_filepath>: Path to private ssh key file

-v, --verbose

--version: Prints version-infos (combinable with -v)

alive

Check if all remote hosts are present & responding.

shepherd-herd alive [OPTIONS]

blink

Help to identify Observers by flashing LEDs near Targets (IO, EMU).

shepherd-herd blink [OPTIONS] [DURATION]

Arguments

DURATION: Optional argument

distribute

Upload a file FILENAME to the remote observers, which will be stored in REMOTE_PATH.

shepherd-herd distribute [OPTIONS] FILENAME

Options

-r, --remote-path <remote_path>: for safety only allowed: /var/shepherd/* or /etc/shepherd/*

-f, --force-overwrite: Overwrite existing file

Arguments

FILENAME: Required argument

emulate

Emulate an energy environment for the attached sensor nodes.

Use the previously recorded harvest-data for emulating an energy environment for the attached sensor nodes and monitor their power consumption and GPIO events - INPUT-PATH is a hdf5-file on the sheep-hosts

shepherd-herd emulate [OPTIONS] INPUT_PATH

Options

-o, --output-path <output_path>: Dir or file path for resulting hdf5 file with load recordings

-d, --duration <duration>: Duration of recording in seconds

-f, --force-overwrite: Overwrite existing file

-c, --use-cal-default: Use default calibration values

--enable-io, --disable-io: Switch the GPIO level converter to targets on/off

--io-port <io_port>

Choose Target that gets connected to IO

Options:: A | B

--pwr-port <pwr_port>

Choose (main)Target that gets connected to virtual Source / current-monitor

Options:: A | B

-x, --voltage-aux <voltage_aux>: Set Voltage of auxiliary Power Source (second target)

-a, --virtual-source <virtual_source>: Use the desired setting for the virtual source

-n, --no-start: Start shepherd synchronized after uploading config

Arguments

INPUT_PATH: Required argument

fix

Reload the shepherd-kernel-module on each sheep.

shepherd-herd fix [OPTIONS]

harvest

Simultaneously record IV data from harvesting-sources on the chosen observers.

shepherd-herd harvest [OPTIONS]

Options

-o, --output-path <output_path>: Dir or file path for resulting hdf5 file

-a, --virtual-harvester <virtual_harvester>: Choose one of the predefined virtual harvesters

-d, --duration <duration>: Duration of recording in seconds

-f, --force-overwrite: Overwrite existing file

-c, --use-cal-default: Use default calibration values

-n, --no-start: Start shepherd synchronized after uploading config

inventorize

Collect information about the observer-hosts -> saved to local file.

shepherd-herd inventorize [OPTIONS] [OUTPUT_PATH]

Arguments

OUTPUT_PATH: Optional argument

poweroff

Power off shepherd observers.

shepherd-herd poweroff [OPTIONS]

Options

-r, --restart: Reboot

program

Programmer for Target-Controller.

shepherd-herd program [OPTIONS] FIRMWARE_FILE

Options

-p, --target-port <target_port>

Choose Target-Port of Cape for programming

Options:: A | B

-m, --mcu-port <mcu_port>: Choose MCU on Target-Port (only valid for SBW & SWD)

-v, --voltage <voltage>: Target supply voltage

-d, --datarate <datarate>: Bit rate of Programmer (bit/s)

-t, --mcu-type <mcu_type>

Target MCU

Options:: nrf52 | msp430

--simulate: dry-run the programmer - no data gets written

Arguments

FIRMWARE_FILE: Required argument

resync

Get current time and restarts PTP on each sheep.

shepherd-herd resync [OPTIONS]

retrieve

Retrieve remote hdf file FILENAME and stores in OUTDIR.

filename: can either be (a) remote file with absolute path or relative path in ‘/var/shepherd/recordings/’ or (b) local job- / task-file that did already run (embedded paths are retrieved)

outdir: local path to put the files in ‘outdir/[node-name]/filename’

shepherd-herd retrieve [OPTIONS] FILENAME OUTDIR

Options

-t, --timestamp: Add current timestamp to measurement file

-s, --separate: Every remote node gets own subdirectory

-d, --delete: Delete the file from the remote filesystem after retrieval

-f, --force-stop: Stop the on-going harvest/emulation process before retrieving the data

Arguments

FILENAME: Required argument

OUTDIR: Required argument

run

Run a task or set of tasks with provided config/task file (YAML).

shepherd-herd run [OPTIONS] CONFIG

Options

-a, --attach: Wait and receive output on shell

Arguments

CONFIG: Required argument

shell-cmd

Run COMMAND on the shell.

shepherd-herd shell-cmd [OPTIONS] COMMAND

Options

-s, --sudo: Run command with sudo

Arguments

COMMAND: Required argument

start

Start pre-configured shp-service.

source /etc/shepherd/config.yml,
UNSYNCED when ‘time_start’ is not set)

shepherd-herd start [OPTIONS]

status

Information about current state of shepherd measurement.

shepherd-herd status [OPTIONS]

stop

Stop any harvest/emulation or other processes blocking the sheep.

shepherd-herd stop [OPTIONS]

Unittests

For testing shepherd-herd there must be a valid herd.yml at one of the mentioned locations (look at Configuration) with accessible sheep-nodes (at least one).

Navigate your host-shell into the package-folder and
install dependencies
run the testbench (~ 30 tests):

cd shepherd/software/shepherd-herd
pip3 install .[tests]
pytest