commit	f4ab0f961bdc4cd396cf205e9c0c58edaccfbbae	[log] [tgz]
author	Rubén Aciego <raciego@netflix.com>	Mon Jan 15 14:16:19 2024 -0800
committer	Joel Sole <jsole@netflix.com>	Tue Mar 26 13:45:54 2024 -0700
tree	cefefeaa98901e51ba230a1ba0c4da943c3ee28a
parent	483fd09bb28a8b060ec958c3a139c30850867cc8 [diff]

tree: cefefeaa98901e51ba230a1ba0c4da943c3ee28a

README.md

AVM

Building

Prerequisites

CMake version 3.16 or higher.
Git.
Perl.
For x86 targets, yasm, which is preferred, or a recent version of nasm. If you download yasm with the intention to work with Visual Studio, please download win32.exe or win64.exe and rename it into yasm.exe. DO NOT download or use vsyasm.exe.
Building the documentation requires doxygen version 1.8.10 or newer.
Building the unit tests requires Python.

Get the code

The AVM project source code is stored in the Alliance of Open Media’s GitLab repository. To get the code,

    $ git clone https://gitlab.com/AOMediaCodec/avm.git
    # By default, the above command stores the source in the avm directory:
    $ cd avm

Basic build

CMake replaces the configure step typical of many projects. Running CMake will produce configuration and build files for the currently selected CMake generator. For most systems the default generator is Unix Makefiles. The basic form of a makefile build is the following:

    $ cmake path/to/avm
    $ make

The above will generate a makefile build that produces the AVM library and applications for the current host system after the make step completes successfully. The compiler chosen varies by host platform, but a general rule applies: On systems where cc and c++ are present in $PATH at the time CMake is run the generated build will use cc and c++ by default.

Configuration options

The AVM codec library has a great many configuration options. These come in two varieties:

Build system configuration options. These have the form ENABLE_FEATURE.
AVM codec configuration options. These have the form CONFIG_FEATURE.

Both types of options are set at the time CMake is run. The following example enables ccache and disables the AVM encoder:

    $ cmake path/to/avm -DENABLE_CCACHE=1 -DCONFIG_MULTITHREAD=0
    $ make

The available configuration options are too numerous to list here. Build system configuration options can be found at the top of the CMakeLists.txt file found in the root of the AVM repository, and AVM codec configuration options can currently be found in the file build/cmake/aom_config_defaults.cmake.

Dylib builds

A dylib (shared object) build of the AVM codec library can be enabled via the CMake built in variable BUILD_SHARED_LIBS:

    $ cmake path/to/avm -DBUILD_SHARED_LIBS=1
    $ make

This is currently only supported on non-Windows targets.

Debugging

Depending on the generator used there are multiple ways of going about debugging AVM components. For single configuration generators like the Unix Makefiles generator, setting CMAKE_BUILD_TYPE to Debug is sufficient:

    $ cmake path/to/avm -DCMAKE_BUILD_TYPE=Debug

For Xcode, mainly because configuration controls for Xcode builds are buried two configuration windows deep and must be set for each subproject within the Xcode IDE individually, CMAKE_CONFIGURATION_TYPES should be set to Debug:

    $ cmake path/to/avm -G Xcode -DCMAKE_CONFIGURATION_TYPES=Debug

For Visual Studio the in-IDE configuration controls should be used. Simply set the IDE project configuration to Debug to allow for stepping through the code.

In addition to the above it can sometimes be useful to debug only C and C++ code. To disable all assembly code and intrinsics set AOM_TARGET_CPU to generic at generation time:

    $ cmake path/to/avm -DAOM_TARGET_CPU=generic

Cross compiling

For the purposes of building the AVM codec and applications, relative to the scope of this guide, all builds for architectures differing from the native host architecture will be considered cross compiles. The AVM CMake build handles cross compiling via the use of toolchain files included in the AVM repository. The available toolchain files can be found at cmake folder in the AVM repository. The following example demonstrates use of the x86-linux.cmake toolchain file on a x86_64 linux host:

    $ cmake path/to/avm \
      -DCMAKE_TOOLCHAIN_FILE=path/to/avm/build/cmake/toolchains/x86-linux.cmake
    $ make

Sanitizers

Sanitizer integration is built-in to the CMake build system. To enable a sanitizer, add -DSANITIZE=<type> to the CMake command line. For example, to enable address sanitizer:

    $ cmake path/to/avm -DSANITIZE=address
    $ make

Sanitizers available vary by platform, target, and compiler. Consult your compiler documentation to determine which, if any, are available.

Microsoft Visual Studio builds

Building the AVM codec library in Microsoft Visual Studio is supported. Visual Studio 2019 (16.7) or later is required. The following example demonstrates generating projects and a solution for the Microsoft IDE:

    # This does not require a bash shell; Command Prompt (cmd.exe) is fine.
    # This assumes the build host is a Windows x64 computer.

    # To build with Visual Studio 2019 for the x64 target:
    $ cmake path/to/avm -G "Visual Studio 16 2019"
    $ cmake --build .

    # To build with Visual Studio 2019 for the 32-bit x86 target:
    $ cmake path/to/avm -G "Visual Studio 16 2019" -A Win32
    $ cmake --build .

NOTE: The build system targets Windows 7 or later by compiling files with -D_WIN32_WINNT=0x0601.

Xcode builds

Building the AVM codec library in Xcode is supported. The following example demonstrates generating an Xcode project:

    $ cmake path/to/avm -G Xcode

Build with VMAF support

After installing libvmaf.a, you can use it with the encoder:

    $ cmake path/to/avm -DCONFIG_TUNE_VMAF=1

Please note that the default VMAF model will be used unless you set the following flag when running the encoder:

    # --vmaf-model-path=path/to/model

Testing

Testing basics

There are several methods of testing the AVM codec. All of these methods require the presence of the AVM source code and a working build of the AVM library and applications.

1. Unit tests:

The unit tests can be run at build time:

    # Before running the make command the LIBAOM_TEST_DATA_PATH environment
    # variable should be set to avoid downloading the test files to the
    # cmake build configuration directory.
    $ cmake path/to/avm
    # Note: The AVM CMake build creates many test targets. Running make
    # with multiple jobs will speed up the test run significantly.
    $ make runtests

2. Example tests:

The example tests require a bash shell and can be run in the following manner:

    # See the note above about LIBAOM_TEST_DATA_PATH above.
    $ cmake path/to/avm
    $ make
    # It's best to build the testdata target using many make jobs.
    # Running it like this will verify and download (if necessary)
    # one at a time, which takes a while.
    $ make testdata
    $ path/to/avm/test/examples.sh --bin-path examples

Downloading the test data

The fastest and easiest way to obtain the test data is to use CMake to generate a build using the Unix Makefiles generator, and then to build only the testdata rule:

    $ cmake path/to/avm -G "Unix Makefiles"
    # 28 is used because there are 28 test files as of this writing.
    $ make -j28 testdata

The above make command will only download and verify the test data.

Additional input data for testing the encoder can be obtained from: AV2 - CTC

Sharded testing

The AVM codec library unit tests are built upon gtest which supports sharding of test jobs. Sharded tests can be achieved as follows for example:

   # Set the environment variable GTEST_TOTAL_SHARDS to control the number of
   # shards.
   $ export GTEST_TOTAL_SHARDS=10
   # (GTEST shard indexing is 0 based).
   $ seq 0 $(( $GTEST_TOTAL_SHARDS - 1 )) \
       | xargs -n 1 -P 0 -I{} env GTEST_SHARD_INDEX={} ./test_libaom

To create a test shard for each CPU core available on the current system set GTEST_TOTAL_SHARDS to the number of CPU cores on your system minus one. The maximum number of test targets that can run concurrently is determined by the number of CPUs on the system where the build is configured as detected by CMake. A system with 24 cores can run 24 test shards using a value of 24 with the -j parameter. When CMake is unable to detect the number of cores 10 shards is the default maximum value.

Coding style

We are using the Google C Coding Style defined by the Google C++ Style Guide.

The coding style used by this project is enforced with clang-format and cmake-format.

clang-format can be installed with your system's package manager, or directly from llvm.org. For best results, your version should match the one used by Gitlab CI, noted as a comment in .clang-format.
cmake-format can be obtained by installing cmakelang. Again, for best results, your version should match the one used by Gitlab CI, noted as a comment in .cmake-format.py

Before pushing changes for review, format your code using the tools above.

We recommend automating the formatting by adding a pre-commit hook at .git/hooks/pre-commit. An example pre-commit hook is provided below:

#!/bin/bash
# Replace 'VERSION' below to match clang-format version in the CI.
CLANG_FORMAT_PATH=/usr/lib/llvm-VERSION/bin
CMAKE_FORMAT_PATH=${HOME}/.local/bin/

echo "Applying clang-format ..."
for file in $(git diff-index --cached --name-only HEAD -- "*.[hc]pp" "*.cc" "*.[ch]") ; do
  ${CLANG_FORMAT_PATH}/clang-format -i --style=file ${file}
  git add ${file}
  echo "Formatted file: $file"
done
echo "Done."

echo "Applying cmake-format ..."
for file in $(git diff-index --cached --name-only HEAD -- '*.cmake' CMakeLists.txt) ; do
  ${CMAKE_FORMAT_PATH}/cmake-format -i ${file}
  git add ${file}
  echo "Formatted file: $file"
done
echo "Done."

Submitting patches

We manage the submission of patches using Gitlab's merge request process. This tool implements a workflow on top of the Git version control system to ensure that all changes get peer reviewed and tested prior to their distribution. If you are not able to submit an MR, please contact SW coordinators to make sure necessary contributor agreements are signed for the AOMedia Project.

Follow the one-time set-up steps as detailed here.
For pushing your code modifications, follow the steps detailed here.
Once the code is pushed into a branch, create a merge request as detailed here.
- The code review, approval and CI/CD process will be initiated after the MR is created.
- Once the MR is approved, the software co-ordinators will merge the MR to the main branch.

Follow the Merge request page to check the status of the changes, review comments etc.

Support

This library is an open source project supported by its community. Please email https://aomedia.org/contact/ for help.

Bug reports

Bug reports can be filed in the Alliance for Open Media Gitlab issue tracker.