Converting Files To C++ Byte Arrays

Occasionally, you may want to include the content of your files in your code. Doing so will essentially ship whatever binary content you included with your binary, removing the need to also distribute files with your programs. In this post, we look at how to convert files to C++ hexadecimal arrays, using built-in terminal tools.

Why Include Files As Hexadecimal Byte Arrays In Your Code?

Long story short, making a binary blob for the contents of a file is a perfectly valid and fast way to use the binary contents of the file in your code.

For example, you may want to store a small icon image for your GUI application in your code, and not ship that image separately. Similarly, for 3D graphics applications, you may want to include the contents of your shader files in a C++ binary array, so you can easily feed it to OpenGL/Vulkan without having to read files from the disk.

Transforming Files To Hexadecimal C++ Arrays

The code block below demonstrates this idea for a one-by-one pixel png image. Specifically, we have the output from hexdump -C of a very small png image.

89 50 4e 47 0d 0a 1a 0a 00 00 00 0d 49 48 44 52
00 00 00 01 00 00 00 01 01 03 00 00 00 25 db 56
ca 00 00 00 03 50 4c 54 45 00 00 00 a7 7a 3d da
00 00 00 01 74 52 4e 53 00 40 e6 d8 66 00 00 00
0a 49 44 41 54 08 d7 63 60 00 00 00 02 00 01 e2
21 bc 33 00 00 00 00 49 45 4e 44 ae 42 60 82

Following that, we want to transform this hexdump into an array we can actually include in our program’s code. Something like the following code block.

unsigned char image[] = {
  0x89, 0x50, 0x4e, 0x47, 0x0d, 0x0a, 0x1a, 0x0a, 0x00, 0x00, 0x00, 0x0d,
  0x49, 0x48, 0x44, 0x52, 0x00, 0x00, 0x00, 0x01, 0x00, 0x00, 0x00, 0x01,
  0x01, 0x03, 0x00, 0x00, 0x00, 0x25, 0xdb, 0x56, 0xca, 0x00, 0x00, 0x00,
  0x03, 0x50, 0x4c, 0x54, 0x45, 0x00, 0x00, 0x00, 0xa7, 0x7a, 0x3d, 0xda,
  0x00, 0x00, 0x00, 0x01, 0x74, 0x52, 0x4e, 0x53, 0x00, 0x40, 0xe6, 0xd8,
  0x66, 0x00, 0x00, 0x00, 0x0a, 0x49, 0x44, 0x41, 0x54, 0x08, 0xd7, 0x63,
  0x60, 0x00, 0x00, 0x00, 0x02, 0x00, 0x01, 0xe2, 0x21, 0xbc, 0x33, 0x00,
  0x00, 0x00, 0x00, 0x49, 0x45, 0x4e, 0x44, 0xae, 0x42, 0x60, 0x82
};
unsigned int image_len = 95;

In other words, we transformed the contents of out 1x1.png file into a C/C++ array we can actually include and use in our code.

Video – Converting Files To Hexadecimal Arrays

In case you’re a visual learner, the video below shows how to use the terminal tool xxd to convert any file to a hexadecimal static array.

Using xxd To Convert Files To C++ Byte Arrays

If you haven’t seen the video in the previous section, the trick for quickly turning files to hexadecimal arrays is using xxd. For a full specification, check out the xxd man page.

In particular, xxd is essentially a tool for making (or reversing) hexadecimal dumps. Specifically, thexxd -i FILE command will turn the file path FILE into a c-style array.

If we use xxd -i with the very small png we previously mentioned you would get the following.

unsigned char image[] = {
  0x89, 0x50, 0x4e, 0x47, 0x0d, 0x0a, 0x1a, 0x0a, 0x00, 0x00, 0x00, 0x0d,
  0x49, 0x48, 0x44, 0x52, 0x00, 0x00, 0x00, 0x01, 0x00, 0x00, 0x00, 0x01,
  0x01, 0x03, 0x00, 0x00, 0x00, 0x25, 0xdb, 0x56, 0xca, 0x00, 0x00, 0x00,
  0x03, 0x50, 0x4c, 0x54, 0x45, 0x00, 0x00, 0x00, 0xa7, 0x7a, 0x3d, 0xda,
  0x00, 0x00, 0x00, 0x01, 0x74, 0x52, 0x4e, 0x53, 0x00, 0x40, 0xe6, 0xd8,
  0x66, 0x00, 0x00, 0x00, 0x0a, 0x49, 0x44, 0x41, 0x54, 0x08, 0xd7, 0x63,
  0x60, 0x00, 0x00, 0x00, 0x02, 0x00, 0x01, 0xe2, 0x21, 0xbc, 0x33, 0x00,
  0x00, 0x00, 0x00, 0x49, 0x45, 0x4e, 0x44, 0xae, 0x42, 0x60, 0x82
};
unsigned int image_len = 95;

From here, you can pretty much copy and paste the output of the command into your source code. But beware that there are better (and more modern) ways to include byte-arrays in your C++ code. We talk about the improvements in the following sections.

Using static constexpr std::array To Improve Your Byte-Arrays

Although the output from xxd -i FILE is perfectly valid C++ code, it is really a C-array to be included in C source code.

If you’ve got a C++ codebase, I’d recommend using std::array to store the data. For example, the following code block shows how to turn our previous C-array into an std::array.

#include <array>
#include <cstdint>

static constexpr std::array<std::uint8_t, 95> image {
  0x89, 0x50, 0x4e, 0x47, 0x0d, 0x0a, 0x1a, 0x0a, 0x00, 0x00, 0x00, 0x0d,
  0x49, 0x48, 0x44, 0x52, 0x00, 0x00, 0x00, 0x01, 0x00, 0x00, 0x00, 0x01,
  0x01, 0x03, 0x00, 0x00, 0x00, 0x25, 0xdb, 0x56, 0xca, 0x00, 0x00, 0x00,
  0x03, 0x50, 0x4c, 0x54, 0x45, 0x00, 0x00, 0x00, 0xa7, 0x7a, 0x3d, 0xda,
  0x00, 0x00, 0x00, 0x01, 0x74, 0x52, 0x4e, 0x53, 0x00, 0x40, 0xe6, 0xd8,
  0x66, 0x00, 0x00, 0x00, 0x0a, 0x49, 0x44, 0x41, 0x54, 0x08, 0xd7, 0x63,
  0x60, 0x00, 0x00, 0x00, 0x02, 0x00, 0x01, 0xe2, 0x21, 0xbc, 0x33, 0x00,
  0x00, 0x00, 0x00, 0x49, 0x45, 0x4e, 0x44, 0xae, 0x42, 0x60, 0x82
};

As you can see, we use static constexpr std::array , to make sure the array is fully constructed at compile time. In addition, the size is the same as what was returned from xxd -i.

However, using a constexpr std::array means that you will not be able to change the contents of the array at runtime. If you require manipulating these bytes at runtime, you can either copy them to a different variable or usestd::vector. for more information, check out my post on C++ vectors and how to use them.

Why Should We Use std::array Instead Of C-Style Arrays?

Before I go on about the benefits of containers compared to the C-arrays, I’d like to say that C-arrays are perfectly valid if your codebase is entirely C.

However, if you’re writing C++, you should know that there are many advantages to using containers (like std::array and std::vector ). Some of them are outlined in the list below.

• Containers can be used with the standard library functions, due to having the begin() and end() functions defined.
• Compilers are very good at optimising containers. In fact, if you’re using std::array to store your file’s bytes, the compiled code should be exactly the same as using c-style arrays.
• Compilers have useful helper functions, such as .size() and .data().

Should You Always Use Hexadecimal Arrays To Include Files In Your Program?

No.

I’d only recommend converting your files into std::array or std::vector of hexadecimal bytes if you have very small files, and the content of these files will not change until you recompile your program again.

If you need to read files that frequently on the disk, you will simply have to use a file stream to read them. Unfortunately.

---

Have I missed anything? Do you have any questions? Useful feedback? Feel free to post a comment below and I will reply as soon as I can!