Signal Logging (Streaming)¶
This document explains how to use the streaming logging features in the firmware.
Important
Streaming is only available when using the Ethernet physical link to the AMDC!
The instructions pick up where the Signal Logging document left off. Before starting the instructions below, make sure you have completed the common logging steps desribed in Signal Logging.
Attention
The streaming logging instructions below assume you have already:
Instrumented your C-code for logging
Registered the variables via the Python logging class
Introduction¶
The streaming logging method is designed specifically for real-time signal plotting and data logging. To that end, the entire streaming interface is wrapped in a real-time plotting class which manages all streaming-related code.
The real-time plotting library is called AMDC_LivePlot.py
.
It is is available in the AMDC-Firmware/scripts/
folder.
AMDC_LivePlot.py
¶
The AMDC_LivePlot
class wraps the streaming interface to the AMDC.
Each instance of the AMDC_LivePlot
class maps to a single variable in the AMDC firmware.
The class handles two main things:
Streaming socket interface from the AMDC where data signals flow
Real-time plotting built on top of the
matplotlib
animation capability
The user does not need to be aware of how the class works. The class can automatically create a streaming socket, capture the data, and plot it – all in real-time.
Python Interface for Streaming Logging¶
Using the streaming-based logging features is completely encapsulated via the Python class AMDC_LivePlot.py
which is available in the AMDC-Firmware/scripts/
folder.
1. Register variables¶
This step should have already been performed, as explained in Signal Logging.
Note
For streaming data, you should not start a buffered log. Simply registering the variables is enough.
2. Import the class¶
from AMDC_LivePlot import AMDC_LivePlot
3. Change Jupyter Notebook/Lab to support interactive plotting¶
The live plotting capabilities require interactive plots in Jupyter. This is enabled by running a magic command prior to creating plots.
If using Jupyter Notebook, run the following magic:
%matplotlib notebook
If using Jupyter Lab, run this magic instead:
%matplotlib widget
Attention
If you receive an error when running your magic command
Confirm whether you are using Jupyter Notebook or Lab (see here)
Ensure that you have installed
ipympl
Exit Jupyter, close all notebook sessions, and re-open.
To go back to static inline plots:
%matplotlib inline
4. Create an AMDC_LivePlot
object¶
To stream variable LOG_foo
from the AMDC:
plot = AMDC_LivePlot(logger, 'foo')
The constructor above takes two optional arguments:
update_interval_ms
– how often to redraw thematplotlib
figure (default is100
)window_sec
– “look-back” period for plotting data; data before the window is discarded (default is1
)
The default arguments above redraw the plot at 10 Hz and keep the last one second of data on the plot.
Attention
After creating the AMDC_LivePlot
object, the graph will immediately start drawing and updating.
However, no data will appear since the actual data streaming is decoupled from the plotting.
5. Start streaming data from the AMDC¶
The AMDC_LivePlot
is ready to start receiving data and plotting it.
Start streaming data via:
plot.start_stream()
After calling this method, the plot should immediately start showing data.
After window_sec
time has elapsed, the x-axis of the plot will start scrolling and old data will be removed.
The latest data is always on the far right of the plot.
The x-axis shows the number of seconds that the AMDC has been running. It will continually increase until the seconds variable wraps in the firmware. The wrapping is not supported in the plotting library.
6. Stop streaming data from the AMDC¶
The AMDC will continuously stream data until it is told to stop:
plot.stop_stream()
The user must manage the starting and stopping of data streaming themselves! If the interactive plot is closed, data streaming will still occur in the background.
Caution
Even if the Python kernel is restarted, the AMDC will still stream data, resulting in buffer overruns since the host is not reading in data. Make sure to stop streaming before stopping the kernel.
Performance¶
A total of four streams are available at one time from the AMDC.
Since each AMDC_LivePlot
only supports a single variable, this means that only four variables can be streamed at once.
The AMDC firmware and host Python library support data streaming at the full sample rate possible: 10 kHz.
However, be aware that the matplotlib
plot is completely redrawn each update.
Plotting a one second window period with 10k data points is slow, so the refresh rate will be limited.
For responsive plots, either reduce the sampling rate (e.g., try 10-100 Hz) or reduce the window period (e.g., try 10 ms).
Note
The performance limitations above are not hard limits; they are simply due to the code implementation.
Future improvements are planned which will support an arbitrary number of variables per stream, and any number of streams.
The true limit is simply the ~1 Gb/s throughput of the Ethernet link (likely already reduced due to the Python socket
library overheads; probably more than 200 Mb/s but less than 1 Gb/s).
For plotting performance, future improvments will implement blitting which will drastically improve plot update rate.
Copy-Paste Example¶
This example shows how to stream the built-in log variables from the blink
user app.
It is assumed to be run in a Jupyter notebook; each cell is marked via comments in the code.
##### CELL START #####
%matplotlib notebook
##### CELL START #####
from AMDC import AMDC
from AMDC_Logger import AMDC_Logger, find_mapfile
from AMDC_LivePlot import AMDC_LivePlot
##### CELL START #####
USE_ETHERNET = True
if not USE_ETHERNET:
amdc = AMDC()
amdc.setup_comm_defaults('uart')
amdc.uart_init('COM6')
else:
amdc = AMDC()
amdc.setup_comm_defaults('eth')
amdc.eth_init()
s0, s0_id = amdc.eth_new_socket('ascii_cmd')
amdc.eth_set_default_ascii_cmd_socket(s0)
##### CELL START #####
mfpath = find_mapfile(r'C:\Users\Nathan\Documents\GitHub\AMDC-Firmware\sdk\app_cpu1')
logger = AMDC_Logger(amdc, mfpath)
##### CELL START #####
logger.sync()
logger.info()
##### CELL START #####
logger.unregister_all()
samples_per_sec = 100
logger.register('vsi_a', var_type = 'double', samples_per_sec = samples_per_sec)
logger.register('vsi_b', var_type = 'float', samples_per_sec = samples_per_sec)
logger.register('vsi_c', var_type = 'int', samples_per_sec = samples_per_sec)
##### CELL START #####
p1 = AMDC_LivePlot(logger, 'vsi_a', window_sec = 1)
p1.start_stream()
p1.show()
##### CELL START #####
p2 = AMDC_LivePlot(logger, 'vsi_b', window_sec = 1)
p2.start_stream()
p2.show()
##### CELL START #####
p3 = AMDC_LivePlot(logger, 'vsi_c', window_sec = 1)
p3.start_stream()
p3.show()
##### CELL START #####
p1.stop_stream()
p2.stop_stream()
p3.stop_stream()