CWG-E120-v1: Implementation ParaKit software on AVM research-v7.0.0, minor updates in setup scripts
updated setup scripts, requirements.txt and README.md
diff --git a/ParaKit/README.md b/ParaKit/README.md
index a45e1a6..1d23342 100644
--- a/ParaKit/README.md
+++ b/ParaKit/README.md
@@ -81,6 +81,8 @@
<b>Step 5:</b> use the generated tables from the `Context-Table_*.h` file under `results/` by copying them into the AVM codebase for testing.
+<b>Rerunning instructions:</b> To be able to run the training on a new dataset or to rerun, it is recommended to delete (or move) the existing data and bistream files. Specifically, developers should delete or move `.csv` files under the `results/data/` folder and the files under `bitstreams/` for a new round of training. Note that the training software expects `bitstreams/` and `results/data/` directories to be present under `ParaKit/`. So, only the csv and bitstreams files under folders should be deleted or moved.
+
---
## 4. Details of configuring parameters.yaml
@@ -108,9 +110,7 @@
---
## 5. Data collection: guidelines for modifying AVM
-The data collection requires some modifications to AVM decoder implementation. For this purpose, `research-v7.0.0-parakit` branch is created as a reference implementation based on AVM.
-
-In the `research-v7.0.0-parakit` branch, the basic data collection module is implemented in `aom_read_symbol_probdata` function by extending the existing `aom_read_symbol` function in AVM. All the changes related to data collection are implemented under the `CONFIG_PARAKIT_COLLECT_DATA` macro. The comments including `@ParaKit` text provides additional information to guide developers on how to extend data collection for different contexts.
+The data collection requires some modifications to AVM decoder implementation. For this purpose, a sample implementation is provided under the `CONFIG_PARAKIT_COLLECT_DATA` macro (disabled by default) as a reference, where the basic data collection module is implemented in `aom_read_symbol_probdata` function by extending the existing `aom_read_symbol` function in AVM. The comments including `@ParaKit` text provides additional information to guide developers on how to extend data collection for different contexts. The macro `CONFIG_PARAKIT_COLLECT_DATA` should be enabled to collect data.
The `research-v7.0.0-parakit` branch implements the necessary changes on top `research-v7.0.0` tag to collect data specifically for `eob_flag_cdf16` and `eob_flag_cdf32` context groups.
Developers can extend this to add support for new (or any other) contexts on by following the changes under `CONFIG_PARAKIT_COLLECT_DATA` macro and instructions in the comments by searching the text `@ParaKit` on their local AVM version.
diff --git a/ParaKit/requirements.txt b/ParaKit/requirements.txt
index b6c9221..baa84a5 100644
--- a/ParaKit/requirements.txt
+++ b/ParaKit/requirements.txt
@@ -1,5 +1,5 @@
-numpy==1.24.4
-pandas==1.2.4
+numpy==1.26.4
+pandas==1.5.3
pandas-stubs==2.2.2.240603
python-dateutil==2.8.2
pytz==2024.1
diff --git a/ParaKit/setup.sh b/ParaKit/setup.sh
index 704e639..0a1b2c2 100644
--- a/ParaKit/setup.sh
+++ b/ParaKit/setup.sh
@@ -1,5 +1,4 @@
-#!/usr/bin/env bash
-set -euo pipefail
+#!/bin/bash
echo "--- Installing ParaKit ---"
if [ ! -d venv ]; then
@@ -11,11 +10,11 @@
python3 -m venv venv
source venv/bin/activate
# update pip
- pip install --upgrade pip
+ python3 -m pip install --upgrade pip
# install required packages
- pip install -r requirements.txt
+ python3 -m pip install -r requirements.txt
# install package locally
- pip install -e .
+ python3 -m pip install -e .
else
echo "venv exists: activating"
source venv/bin/activate
diff --git a/ParaKit/setup_decoder.sh b/ParaKit/setup_decoder.sh
index a97b837..6bd5a66 100644
--- a/ParaKit/setup_decoder.sh
+++ b/ParaKit/setup_decoder.sh
@@ -1,11 +1,9 @@
-#!/usr/bin/env bash
-set -euo pipefail
+#!/bin/bash
#create directories
mkdir -p binaries
if [ ! -f binaries/aomdec ]; then
-
#check if AVM directory is available based on LICENSE file
is_parent_avm=$(cat ../LICENSE | grep "Alliance for Open Media")
if [ -n "$is_parent_avm" ]; then
diff --git a/build/cmake/aom_config_defaults.cmake b/build/cmake/aom_config_defaults.cmake
index af6c50b..572556d 100644
--- a/build/cmake/aom_config_defaults.cmake
+++ b/build/cmake/aom_config_defaults.cmake
@@ -77,7 +77,7 @@
set_aom_config_var(CONFIG_GPROF 0 "Enable gprof support.")
set_aom_config_var(CONFIG_LIBYUV 1 "Enables libyuv scaling/conversion support.")
-set_aom_config_var(CONFIG_MULTITHREAD 0 "Multithread support.")
+set_aom_config_var(CONFIG_MULTITHREAD 1 "Multithread support.")
set_aom_config_var(CONFIG_OS_SUPPORT 0 "Internal flag.")
set_aom_config_var(CONFIG_PIC 0 "Build with PIC enabled.")
set_aom_config_var(CONFIG_RUNTIME_CPU_DETECT 1 "Runtime CPU detection support.")
@@ -139,7 +139,7 @@
set_aom_config_var(CONFIG_ZERO_OFFSET_BITUPSHIFT 1
"Use zero offset for non-normative bit upshift")
-set_aom_config_var(CONFIG_PARAKIT_COLLECT_DATA 1
+set_aom_config_var(CONFIG_PARAKIT_COLLECT_DATA 0
"enables data collection for ParaKit training.")
# AV2 experiment flags.
