Install Precompiled TensorFlow Lite on Raspberry Pi

Install Precompiled TensorFlow Lite on Raspberry Pi

TensorFlow Lite is an open-source library designed to run machine learning models and perform inference on edge devices, such as mobile and embedded devices. TensorFlow Lite does not support model training; instead, models must be trained using TensorFlow or another framework before deployment. Prior to inference, a TensorFlow model must be converted into a TensorFlow Lite model using the TensorFlow Lite Converter. This tutorial shows how to install precompiled TensorFlow Lite on Raspberry Pi.

Install TensorFlow Lite

Use SSH to connect to Raspberry Pi. Execute the following command to download the .deb package from releases page of the repository:

curl -sSLo tensorflow-lite.deb https://github.com/prepkg/tensorflow-lite-raspberrypi/releases/latest/download/tensorflow-lite-aarch64-linux-gnu.deb

When the download is finished, install TensorFlow Lite:

sudo apt install -y ./tensorflow-lite.deb

The .deb package is no longer necessary, remove it:

rm -rf tensorflow-lite.deb

Testing TensorFlow Lite (C API)

The Debian package contains shared libraries for the C and C++ APIs. The C API will be tested first. Before starting the test, install the GNU C compiler:

sudo apt install -y gcc

For testing, a TensorFlow Lite model is required. The model can be obtained by following the post on converting a TensorFlow 2 model to a TensorFlow Lite model, or by downloading a pre-trained TensorFlow Lite model from the Internet:

curl -sSLo model.tflite https://www.dropbox.com/s/b1426ewx13idlr0/simple_linear_regression.tflite?dl=1

This model solves the simple linear regression problem described in the post.

Create a main.c file:

nano main.c

Add the following code:

#include <stdio.h>
#include <tensorflow/lite/c/common.h>
#include <tensorflow/lite/c/c_api.h>

int main(void) {
    int numThreads = 4;

    TfLiteModel *model = TfLiteModelCreateFromFile("model.tflite");

    TfLiteInterpreterOptions *options = TfLiteInterpreterOptionsCreate();
    TfLiteInterpreterOptionsSetNumThreads(options, numThreads);
    TfLiteInterpreter *interpreter = TfLiteInterpreterCreate(model, options);

    TfLiteInterpreterAllocateTensors(interpreter);

    float x[] = {15.0f};

    TfLiteTensor *inputTensor = TfLiteInterpreterGetInputTensor(interpreter, 0);
    TfLiteTensorCopyFromBuffer(inputTensor, x, sizeof(x));

    TfLiteInterpreterInvoke(interpreter);

    float y[1];

    const TfLiteTensor *outputTensor = TfLiteInterpreterGetOutputTensor(interpreter, 0);
    TfLiteTensorCopyToBuffer(outputTensor, y, sizeof(y));

    printf("%.4f\n", y[0]);

    TfLiteInterpreterDelete(interpreter);
    TfLiteInterpreterOptionsDelete(options);
    TfLiteModelDelete(model);

    return 0;
}

A code is used to predict a value of y for a previously unknown value of x. The model was trained using the following relationship between variables: y = 2 * x + 1.

The model is loaded, and the TensorFlow Lite interpreter is initialized. The value of the x variable is copied into the input tensor buffer, and the model is executed. The value from the output tensor buffer is then copied into the y variable. Finally, the result is printed, and the allocated resources are released.

Compile a code:

gcc main.c -o test -ltensorflowlite_c

Run a program:

./test

In this case, x is 15.0 and model returns that y is 31.0044. The result can be verified:

y = 2 * x + 1 = 2 * 15 + 1 = 31

Testing TensorFlow Lite (C++ API)

The C API can be used within C++ code. However, TensorFlow Lite also provides a dedicated C++ API. Make sure that the GNU C++ compiler is installed:

sudo apt install -y g++

Create a main.cpp file:

nano main.cpp

When a file is opened, add the following code:

#include <iostream>
#include <tensorflow/lite/interpreter.h>
#include <tensorflow/lite/kernels/register.h>

using namespace tflite;

int main() {
    int numThreads = 4;

    std::unique_ptr<FlatBufferModel> model = FlatBufferModel::BuildFromFile("model.tflite");

    ops::builtin::BuiltinOpResolver resolver;
    std::unique_ptr<Interpreter> interpreter;
    InterpreterBuilder(*model, resolver)(&interpreter, numThreads);

    interpreter->AllocateTensors();

    float x[] = {15.0f};

    float *inputTensor = interpreter->typed_input_tensor<float>(0);
    memcpy(inputTensor, x, sizeof(x));

    interpreter->Invoke();

    float *y = interpreter->typed_output_tensor<float>(0);

    std::cout << y[0] << std::endl;

    return 0;
}

The code was implemented using the C++ API and performs the same functionality as the code implemented with the C API.

Execute the following command to compile a code:

g++ main.cpp -o test -ltensorflow-lite

Run a program:

./test

Uninstall TensorFlow Lite

If decided to completely remove TensorFlow Lite, run the following command:

sudo apt purge --autoremove -y tensorflow-lite

Related

The 20 Comments Found

  1. Avatar
    Sanjib Acharya Reply

    Hi,
    Thanks for posting. I was trying to follow the instructions to install Tensorflow-Lite on my RPi 4b for the use of libcamera-detect. After I install using the first two commands, I do not see any tensorflow-lite folder in my /home/pi directory. Where is this installed? During installation, I see messages saying" unpacking and setting up tensorflow-lite (2.7.0-2):

    pi@raspberrypi:~ $ sudo apt install -y ./tensorflow-lite.deb
Reading package lists... Done
Building dependency tree       
Reading state information... Done
Note, selecting 'tensorflow-lite' instead of './tensorflow-lite.deb'
The following NEW packages will be installed:
  tensorflow-lite
0 upgraded, 1 newly installed, 0 to remove and 0 not upgraded.
Need to get 0 B/1,608 kB of archives.
After this operation, 0 B of additional disk space will be used.
Get:1 /home/pi/tensorflow-lite.deb tensorflow-lite armhf 2.7.0-2 [1,608 kB]
Selecting previously unselected package tensorflow-lite.
(Reading database ... 133705 files and directories currently installed.)
Preparing to unpack /home/pi/tensorflow-lite.deb ...
Unpacking tensorflow-lite (2.7.0-2) ...
Setting up tensorflow-lite (2.7.0-2) ...

    But can't see where it is installed. Also, when I run the test as mentioned in this page, it fails with an error:

    pi@raspberrypi:~ $ g++ main.cpp -o test -ltensorflow-lite -ldl
/usr/bin/ld: //usr/local/lib/libtensorflow-lite.so: undefined reference to `log@GLIBC_2.29`
/usr/bin/ld: //usr/local/lib/libtensorflow-lite.so: undefined reference to `exp@GLIBC_2.29`
/usr/bin/ld: //usr/local/lib/libtensorflow-lite.so: undefined reference to `pow@GLIBC_2.29`
collect2: error: ld returned 1 exit status

    What might be wrong?

    • Avatar
      lindevs Reply

      Hi, Sanjib
      Shared libraries libtensorflow-lite.so and libtensorflowlite_c.so are installed to /usr/local/lib directory:
      ls /usr/local/lib | grep tensorflow

      Header files are installed to /usr/local/include/tensorflow directory:
      ls /usr/local/include/tensorflow

      I see that you are using older version of Raspberry Pi OS. At this moment precompiled TensorFlow Lite shared libraries are compatible with Raspberry Pi OS Bullseye. So you need to update Raspberry Pi OS in order to use the library.

    • Avatar
      lindevs Reply

      Hi
      It only includes C and C++ libraries:
      libtensorflowlite_c.so (C library)
      libtensorflow-lite.so (C++ library)

    • Avatar
      lindevs Reply

      Hi, Rahul
      The tensorflow-lite.deb package contains the FlatBuffers header files which are installed to /usr/local/include/flatbuffers directory:
      ls /usr/local/include/flatbuffers

  4. Avatar
    ThomasD Reply

    Hello,
    with a Raspberry B+ V1.2 I get an error message with GCC and G++: "Illegal instruction"
    regards

    • Avatar
      lindevs Reply

      Hi,
      Precompiled TensorFlow Lite libraries are only compatible with Raspberry Pi 3 Model A+/B+ and Raspberry Pi 4 Model B.

  5. Avatar
    William A Reply

    Your Tensorflow binaries are working great. I tried several times to install Tensorflow on my own and was never able to get my programs to compile. This was so quick and easy.

    Thank you!!!

  6. Avatar
    adam Reply

    This took me a few hours to workout.... When building rpicam-apps for tensorflow and you get the error "tensorflow-lite" not found, tried pkgconfig and cmake.." then you might be missing 'tensorflow-lite.pc'. Create the file in '/usr/local/lib/pkgconfig' with the following content.

    prefix=/usr/local
exec_prefix=${prefix}
libdir=${exec_prefix}/lib
includedir=${prefix}/include

Name: TensorFlow Lite
Description: TensorFlow Lite library
Version: 2.6.0
Libs: -L${libdir} -ltensorflow-lite
Cflags: -I${includedir}
    • Avatar
      bruce Reply

      Hi Adam,
      Thank you for the advice Adam. I had the same problem you had!

      Note the the file 'tensorflow-lite.pc' with the specified contents needs to be placed within the folder '/usr/local/lib/pkgconfig'. I missed that - assumed the filename had to be 'pkgconfig'... :((

  7. Avatar
    HAKAN Reply

    Reading package lists... Done
    Building dependency tree... Done
    Reading state information... Done
    Note, selecting 'tensorflow-lite:arm64' instead of './tensorflow-lite_64.deb'
    Some packages could not be installed. This may mean that you have
    requested an impossible situation or if you are using the unstable
    distribution that some required packages have not yet been created
    or been moved out of Incoming.
    The following information may help to resolve the situation:

    The following packages have unmet dependencies:
    tensorflow-lite:arm64 : Depends: python3-numpy:arm64 (>= 1.24.2) but it is not installable

    • Avatar
      lindevs Reply

      Hi,
      It's likely that you've installed an older version of Raspberry Pi OS. Make sure to install the latest version, Raspberry Pi OS Bookworm. Also, verify that you're using the 64-bit version of the OS.

  8. Avatar
    GarryM Reply

    I would be interested in how you build the shared libraries as I had tried a number of times without success

  9. Avatar
    Shadab Reply

    Hey,
    I am trying to build TFLite for linux on amd64 architecture. Is it possible for you to post how did you build it for raspberry pi architecture?

    • Avatar
      lindevs Reply

      Hi,
      I plan to release build scripts on GitHub repository in the future. Please note that precompiling packages for the Raspberry Pi involves a different process compared to compiling packages directly on an amd64 architecture.

      • Avatar
        Shadab Reply

        Thank you, I do understand that there will be differences in the build process but it will be helpful to try understand the steps and implement the build for amd64 architecture.

  10. Avatar
    Artur Reply

    Thank you so much for updating this, I've had plenty of trouble building 2.19 (I'm not good with C++)

Leave a Comment

Cancel reply

Your email address will not be published.