Baseline solutions

Two baseline approaches are provided together with the devkit. These baselines will help facilitate the assessment of more advanced AI solutions and help familiarize the participants with the dataset, methodologies, and evaluation pipeline. Participants who are new to the field can leverage the baseline implementation as a starting point and build upon the baseline implementation by iterating on the existing model and experimenting with modifications and incorporating newer AI techniques.

Heuristic-based approach

The Satellite Node Identification and Classification Tool (SNICT) is provided as a heuristic-based method of characterizing the station-keeping and propulsion features of a satellite’s PoL by tracking changes in the satellite’s longitude and inclination waveforms. These changes are identified from variations in the standard deviation per orbital period of the waveform. SNICT examines the value of this localized standard deviation at each two-hour timestep within the six-month study period and detects events when this value crosses a static threshold—chosen to reflect the most commonly observed maximum value exhibited by typical station-keeping behavioral modes.

A low-to-high crossing indicates a potential initiate drift (ID) node, suggesting that the satellite has transitioned from a station-keeping behavioral mode to a drifting behavioral mode. Conversely, high-to-low crossings indicate potential initiate station-keeping (IK) nodes where the satellite has entered a station-keeping behavioral mode after a transitional (non-station-keeping) mode—such as a period of natural drift. An additional adjust drift (AD) node type is detected for the in-track direction when the local standard deviation of the longitude waveform changes by more than 10% during a transitional behavioral mode. When all potential nodes within the study period are identified, they are filtered for false detections resulting from transient noise.

The propulsion features of any station-keeping behavioral modes are characterized after node-detection has concluded. Since SNICT does not detect or characterize individual maneuvers, the propulsion type is assigned by a function of the longitude’s non-localized standard deviation and the size of its deadband between the bounding IK and ID nodes. In-track and cross-track station-keeping is always assumed to utilize the same propulsion type.

Machine learning-based approach

A random forest classifier model was developed as the machine learning baseline approach for this challenge. The model takes in the multi-dimensional satellite state data and predicts the PoL node labels at each time step. To enable classic tabular machine learning, the model has to be trained to predict labels at each time step (row in the table), but its performance must be evaluated based on change point detection accuracy compared to the ground truth. Additionally, since there can be more than one node at the same time step (one for EW and one for NS), separate models have to be trained for each direction.

To meet these requirements, the ground truth data is first separated into EW and NS types. The “node” and “type” columns are concatenated to create a single label for both directions EW and NS, such as ‘SS-CK’. These labels are forward filled so each row has a label. Additionally, the input data is expanded by creating lagged features from the raw feature columns (position, velocity, orbital elements, etc). Lagged features capture historical context by using the prior n time steps of each feature. The final data table with labels is then split into training and validation sets by satellite ObjectID to avoid data leakage.

The model predictions are post-processed to convert the per time step predictions into a format matching the ground truth. To do this, the predicted EW and NS labels are first split back into the original “node” and “type” columns. Then, they are filtered to only keep rows where the label changes compared to the prior time step. This process converts both the ground truth and predictions into a consistent format with labels provided only at change points.