The new re­lease brings Python bind­ings, Ba­sis Uni­ver­sal tex­ture com­pres­sion, im­proved STL in­ter­op­er­abil­i­ty, bet­ter Uni­code ex­pe­ri­ence for Win­dows users, a more ef­fi­cient Em­scripten ap­pli­ca­tion im­ple­men­ta­tion, sin­gle-head­er li­braries, new OpenGL driv­er work­arounds and much more.

STL-com­pat­i­ble con­tain­er class­es and fur­ther work on re­duc­ing com­pile times

Con­tin­u­ing from Con­tain­ers::Point­er and Con­tain­ers::Op­tion­al in­tro­duced in the pre­vi­ous re­lease, the 2019.10 re­lease adds STL com­pat­i­bil­i­ty to its Con­tain­ers::Ar­rayView class­es as well. In prac­tice this means std::vec­tor or std::ar­ray in­stances can be im­plic­it­ly con­vert­ed to Mag­num ar­ray views, and if you’re on the bleed­ing edge and use C++2a std::span, the con­ver­sion can go both ways:

#include <Corrade/Containers/ArrayViewStl.h> // enables implicit conversions

/* Image view backed by a std::vector */
std::vector<char> pixels = ;
MutableImageView2D image{PixelFormat::RGBA8Unorm, {32, 24}, pixels};

/* Getting the data back as a span on C++2a */
std::span<char> span = image.data();

The new re­lease al­so in­tro­duces a re­worked Con­tain­ers::StridedAr­rayView, now sup­port­ing mul­ti­ple di­men­sions and ze­ro / neg­a­tive strides, giv­ing it a fea­ture par­i­ty with numpy.ndar­ray in Python. It’s al­so get­ting used in a broad­er set of APIs, in­clud­ing Mesh­Tools::gen­er­ateS­mooth­Nor­mals(), range-tak­ing Math::min() or the very use­ful Im­age::pix­els() that gives raw typed ac­cess to pix­el da­ta, and even print them on ter­mi­nal:

importer.openFile("mosra.png");
Trade::ImageData2D image = importer.image2D(0);
Debug{} << Debug::color << Debug::packed << image.pixels<Color3ub>();
‌▓▓‌▓▓‌▓▓‌▓▓‌▓▓‌▓▓‌▒▒‌░░‌░░‌▓▓‌▓▓‌▓▓‌▓▓‌▓▓
‌▓▓‌▓▓‌▓▓‌▓▓‌▓▓‌▒▒‌░░‌░░‌▓▓‌▓▓‌▓▓‌▓▓
‌▓▓‌▓▓‌▓▓‌▓▓‌▓▓‌░░‌░░‌▓▓‌▓▓‌▓▓‌▓▓
‌▓▓‌▓▓‌▓▓‌▓▓‌░░‌░░‌░░‌▒▒‌▓▓‌▓▓‌▓▓
‌▓▓‌▓▓‌██‌▓▓‌░░‌░░‌▓▓‌▓▓‌▓▓
‌██‌▓▓‌██‌░░‌░░‌▒▒‌▒▒‌▒▒‌▓▓‌░░‌▒▒‌██‌▓▓
‌██‌██‌▓▓‌▒▒‌▓▓‌██‌██‌██‌██‌▓▓‌░░‌▒▒‌▓▓‌██
‌██‌██‌▒▒‌▒▒‌▓▓‌▓▓‌██‌██‌██‌▓▓‌░░‌░░‌▓▓‌██
‌██‌██‌░░‌░░‌▓▓‌▓▓‌██‌██‌██‌██‌▓▓‌░░‌▒▒‌██
‌██‌██‌░░‌░░‌▓▓‌▓▓‌▓▓‌██‌██‌▓▓‌▓▓‌░░‌░░‌▓▓
‌██‌▓▓‌░░‌░░‌▒▒‌▓▓‌▓▓‌▓▓‌▓▓‌▒▒‌░░‌▓▓
‌██‌▓▓‌░░‌░░‌▒▒‌░░‌▓▓
‌██‌▓▓‌▒▒‌▓▓‌░░‌░░‌░░‌▓▓‌▓▓
‌▓▓‌▒▒‌░░‌▓▓‌▒▒‌▒▒‌▒▒‌▓▓‌▓▓‌▒▒
‌▓▓‌░░‌▒▒‌▓▓‌▒▒‌▒▒‌██‌██‌░░‌░░
‌▒▒‌▒▒‌▒▒‌▒▒‌▓▓‌▓▓‌░░
‌░░‌▒▒‌▓▓‌▒▒‌░░‌░░
‌░░‌▒▒‌▓▓‌▓▓‌▒▒‌░░‌░░
‌░░‌░░‌░░‌░░‌░░
‌░░
‌░░‌░░


All this in­ter­op­er­abil­i­ty how­ev­er doesn’t mean all head­ers sud­den­ly need to #include <vector> or the like. On the con­trary — the con­ver­sion is en­abled by in­clud­ing ded­i­cat­ed head­ers list­ed be­low, to­geth­er with the abil­i­ty to for­ward-de­clare some STL types when you don’t need the full def­i­ni­tion. Put all to­geth­er, this means the code both com­piles faster (when you don’t need to use STL types) and can in­ter­op­er­ate with STL types bet­ter (when you ac­tu­al­ly want that).

New pow­er-ef­fi­cient ap­pli­ca­tion im­ple­men­ta­tion for Em­scripten

While all Mag­num ap­pli­ca­tions his­tor­i­cal­ly de­fault­ed to re­draw­ing and mak­ing the CPU busy on­ly when need­ed in or­der to save pow­er, this was not re­al­ly the case on the web. Con­trib­uted by @Squareys, there’s a new Plat­form::Em­scripte­nAp­pli­ca­tion that aims to be more ef­fi­cient and small­er to down­load. Be­sides that, the Em­scripten SDL “em­u­la­tion” has a lot of lim­i­ta­tions and hav­ing an im­ple­men­ta­tion based di­rect­ly off the HTM­L5 APIs al­lows us to be more flex­i­ble.

The new im­ple­men­ta­tion was al­so used for an ex­per­i­ment in how far can Mag­num po­ten­tial­ly get with ex­e­cutable size op­ti­miza­tion. A few of those op­ti­miza­tions al­ready made it to 2019.10 and lots more is in the buf­fer for next re­leas­es — sub­scribe to mosra/mag­num#293 for up­dates.

52.1 kB 36.3 kB 35.7 kB 19.4 kB 14.7 kB 14.7 kB 14.7 kB 14.7 kB 14.7 kB 14.7 kB 226.3 kB 224.5 kB 224.5 kB 224.5 kB 83.6 kB 75.4 kB 69.3 kB 62.6 kB 56.3 kB 44.0 kB 0 50 100 150 200 250 kB Initial state Enabling minimal runtime Additional slimming flags Disabling filesystem Chopping off all C++ stream usage Enabling CORRADE_NO_ASSERT Removing a single use of std::sort() Removing one std::unordered_map Using emmalloc instead of dlmalloc Removing all printf() usage Download size (*.js, *.wasm)

Python bind­in­gs and Eigen in­ter­op­er­abil­i­ty

By far the largest part of this re­lease are the new Python bind­ings, made us­ing py­bind11 and avail­able through a sep­a­rate repos­i­to­ry at https://github.com/mosra/magnum-bindings. To get a first im­pres­sions, check out how the ba­sic C++ tu­to­ri­als look like when rewrit­ten in Python. Large ef­fort went in­to mak­ing the Python API feel like Python, in­clud­ing GLSL-like vec­tor swiz­zles:

>>> from magnum import *
>>> a = Vector4(1.5, 0.3, -1.0, 1.0)
>>> b = Vector4(7.2, 2.3, 1.1, 0.0)
>>> a.wxy = b.xwz
>>> a
Vector(0, 1.1, -1, 7.2)

The bind­ings are op­ti­mized for ze­ro-copy da­ta trans­fer be­tween C++ and Python us­ing CPython’s Buf­fer Pro­to­col, which at the very core means the Mag­num ar­ray view class­es got ex­posed as con­tain­ers.Ar­rayView, con­tain­ers.StridedAr­rayView1D etc., with sup­port for the full Python slic­ing syn­tax and in­ter­op­er­abil­i­ty with numpy.ndar­ray, mem­o­ryview and oth­er views and con­tain­ers.

While NumPy is used ex­ten­sive­ly by re­searchers in the Python world, same could be said about Eigen in the C++ world. In 2019.10, Eigen­In­te­gra­tion joins the ranks of GlmInte­gra­tion in bring­ing builtin con­ver­sion be­tween for­eign and Mag­num math types. Goal for both is not need­ing to wor­ry about whether ma­trix is row- or col­umn-ma­jor or which or­der quater­nion com­po­nents are stored in:

#include <Magnum/EigenIntegration/Integration.h>

Eigen::Vector3f a{1.0f, 2.0f, 3.0f};
Vector3 b(a);

auto c = Matrix4::rotation(Vector3(a), 35.0_degf);

Im­age API im­prove­ments

With Mu­ta­bleIm­ageView2D and friends and new over­loads to GL::Ab­stract­Frame­buf­fer::read(), GL::Tex­ture::im­age() etc. it’s now pos­si­ble to read GPU im­ages in­to ex­ist­ing mem­o­ry, with­out un­want­ed large mem­o­ry al­lo­ca­tions hap­pen­ing in the back­ground. These new APIs are al­so ex­posed to Python, al­low­ing for ef­fi­cient trans­fer of ren­dered im­ages di­rect­ly in­to a mem­o­ry buf­fer man­aged by a ma­chine learn­ing frame­work, for ex­am­ple.

Back in 2018.04, Mag­num gained back­end-in­de­pen­dent pix­el for­mats, how­ev­er the Com­pressed­Pix­elFor­mat enum was quite ne­glect­ed un­til now, sup­port­ing just ba­sic S3TC. Now it sup­ports all wide­ly-used com­pres­sion for­mats — sRGB S3TC vari­ants, one/two-chan­nel BC4 and BC5 for­mats, BC6h and BC7, ETC2 and EAC for­mats for mo­bile plat­forms, ASTC (in­clud­ing 3D and HDR) and PVRTC. On the GL side, GL::Com­pressed­Pix­elFor­mat learned PVRTC for­mats as well, ex­posed the (3D) ASTC for­mats for We­bGL, and same was done for the Vk::vk­For­mat() con­ver­sion util­i­ty. Be­sides GL and Vulkan, the Pix­elFor­mat / Com­pressed­Pix­elFor­mat enum doc­u­men­ta­tion now lists al­so cor­re­spond­ing D3D and Met­al val­ues to make it eas­i­er for peo­ple us­ing (or com­ing from) these back­ends.

These im­prove­ments are the ini­tial batch of new fea­tures be­ing added, with more fol­low­ing next — im­proved DDS sup­port (see mosra/mag­num-plug­ins#67), a KTX2 im­porter or, for ex­am­ple, mip lev­el se­lec­tion (mosra/mag­num#369).

Ba­sis Uni­ver­sal tex­ture com­pres­sion

The main rea­son why all the above-list­ed com­pres­sion for­mats were added is Ba­sis Uni­ver­sal. It’s a suc­ces­sor to Crunch, open-sourced a few months ago thanks to fund­ing from Google. What makes it so rev­o­lu­tion­al is best ex­plained by the fol­low­ing plot. I took the cov­er.jpg used on top of this ar­ti­cle and con­vert­ed it to cov­er.ba­sis and a bunch of raw block com­pres­sion for­mats for com­par­i­son:

215.0 kB 1296.0 kB 269.5 kB 154.9 kB 0.0 kB 0.0 kB 242.1 kB 0.0 kB 0.0 kB 0.0 kB 508.1 kB 0.0 kB 4969.0 kB 0.0 kB 276.3 kB 1141.1 kB 0 1000 2000 3000 4000 5000 kB JPEG -> RGBA8 Uncompressed BC3 DDS -> BC3 Compressed BC3 + ETC2 + PVRTC -> BC3 Basis Universal -> BC3 File size / memory use

Be­fore Ba­sis, you had ba­si­cal­ly two ways how to op­ti­mize your as­set size:

  • Ei­ther op­ti­mize stor­age size by us­ing lossy com­pres­sion (such as JPEG), but then hav­ing to ful­ly un­com­press to RG­BA8. With the 1536×864 cov­er im­age it’s a ~200 kB im­age in­flat­ed to over 5 MB of RG­BA da­ta.
  • Or op­ti­mize GPU mem­o­ry us­age by us­ing var­i­ous block com­pres­sion for­mats (such as BC3 / DX­T5), which is on­ly 1.3 MB of da­ta in mem­o­ry; and with a loss­less com­pres­sion on top you’ll get down to a 270 kB file. How­ev­er, es­pe­cial­ly on mo­bile de­vices, each GPU ven­dor sup­ports a dif­fer­ent for­mat so you need to ship at least a BCn, ETC and PVRTC vari­ant.

With Ba­sis Uni­ver­sal, you get the best of both worlds — da­ta is in­ter­nal­ly stored in a sub­set of the ETC1 block com­pres­sion for­mat with ad­di­tion­al com­pres­sion on top, mak­ing it small­er than an equiv­a­lent JPEG, and then you can transcode that sin­gle file to BCn, ETC2, ASTC or PVRTC de­pend­ing on what the GPU needs.

Thanks to work done by @Squareys, Mag­num sup­ports both im­port­ing (and transcod­ing to a de­sired GPU for­mat) via the Ba­sisIm­porter plug­in as well as en­cod­ing im­ages in­to the Ba­sis Uni­ver­sal for­mat us­ing the Ba­sisIm­age­Con­vert­er. Com­pared to the of­fi­cial basisu tool, which works on­ly with PNGs, the mag­num-im­age­con­vert­er util­i­ty sup­ports any for­mat that Mag­num can im­port:

magnum-imageconverter image.jpg image.basis

Of course, all op­tions sup­port­ed by basisu are ex­posed to the plug­in con­fig­u­ra­tion as well:

magnum-imageconverter image.jpg --converter BasisImageConverter \
    -c flip_y=false,threads=8 image.basis

The TinyGlt­fIm­porter sup­ports Ba­sis files through the un­of­fi­cial GOOGLE_texture_basis ex­ten­sion. There are still some fea­tures we’re wait­ing on to get merged in or­der to have a full sup­port. One of them is an abil­i­ty to Y-flip im­ages dur­ing transcode (in­stead of on­ly in the en­coder, Bi­no­mi­al­LLC/ba­sis_u­ni­ver­sal#79), an­oth­er are buildsys­tem im­prove­ments (Bi­no­mi­al­LLC/ba­sis_u­ni­ver­sal#13) — right now, the soft­ware can’t be built as a li­brary on its own and thus is im­pos­si­ble to pack­age / dis­trib­ute with­out re­quir­ing each project to bun­dle it. Un­til that’s re­solved, Ba­sis won’t be en­abled in any Mag­num pack­ages. The on­ly ex­cep­tion is Vcp­kg, where a Ba­sis fork, based off the above PR, is used.

Mag­num Play­er im­prove­ments

The Mag­num Play­er util­i­ty re­ceived quite a few new fea­tures. It can now au­to­mat­i­cal­ly gen­er­ate smooth nor­mals for mod­els that don’t have them and you can in­spect mesh topol­o­gy by se­lect­ing it us­ing a right-click.

Apart from mesh­es, the play­er can now al­so open im­ages of all types that Mag­num can im­port. This in­cludes the above-men­tioned Ba­sis Uni­ver­sal — and the web ver­sion knows those, too, and transcodes to BCn, ETC, PVRTC, ASTC or plain RG­BA de­pend­ing on what your brows­er sup­ports.

DART in­te­gra­tion and an ex­am­ple

The Dart­In­te­gra­tion li­brary, in­te­grat­ing the DART An­i­ma­tion and Ro­bot­ics Tool­kit, con­trib­uted by @costashatz over a year ago, now re­ceived a well-pol­ished in­ter­ac­tive ex­am­ple. As a side dish, Costas wrote a de­tailed over­view post, ex­plain­ing both the code and the ro­bot­ics back­ground:

Us­ing DART to con­trol a ro­bot­ic ma­nip­u­la­tor »
Gues post by Kon­stanti­nos Chatzi­lyger­oud­is

Buildsys­tem us­abil­i­ty im­prove­ments

The 2019.10 re­lease irons out the re­main­ing pain points in us­ing Mag­num li­braries as CMake sub­pro­jects. All bi­na­ries are now put in­to a com­mon di­rec­to­ry in­side the build dir, which means no has­sle with DLL paths on Win­dows any­more — and to help the com­mon use cas­es even fur­ther, SDL and GLFW DLLs are au­to­mat­i­cal­ly copied there as well.

Plug­in us­age with CMake sub­pro­jects is sig­nif­i­cant­ly im­proved too. Dy­nam­ic plug­in bi­na­ries are put in a cen­tral place in the build di­rec­to­ry and the plug­in man­agers now look for them rel­a­tive­ly to lo­ca­tion of giv­en plug­in in­ter­face li­brary, re­mov­ing the need to in­stall ev­ery­thing first.

set(WITH_TINYGLTFIMPORTER ON)
set(WITH_STBIMAGEIMPORTER ON)
add_subdirectory(magnum-plugins)

Note that the above bumped the min­i­mal CMake ver­sion re­quire­ment from 3.1 to 3.4, al­though we don’t ex­pect any is­sues as the ver­sions cur­rent­ly in wide­spread use is 3.5. In any case, you can al­ways down­load a pre­built ver­sion for your plat­form.

Thanks to ex­ten­sive feed­back from @alan­jfs, the Get­ting Start­ed Guide got rewrit­ten and is now eas­i­er to fol­low by first-time users on Win­dows, not re­quir­ing any­body to fid­dle with %PATH% or in­stalling things to ran­dom places any­more.

Win­dows-spe­cif­ic good­ies

Com­pared to 2019.01, there’s an of­fi­cial sup­port for MSVC 2019. The com­pil­er still needs a few work­arounds com­pared to GCC / Clang, but it’s rel­a­tive­ly mi­nor things that should not af­fect us­abil­i­ty. Ex­trap­o­lat­ing fur­ther, we ex­pect the next ver­sion of MSVC to be ful­ly con­form­ing, and thus not need­ing any com­pil­er-spe­cif­ic han­dling. We’re com­mit­ed to ful­ly sup­port­ing all pre­vi­ous ver­sions back to MSVC 2015 for the fore­see­able fu­ture.

Cor­rade::Main is a new li­brary that, on Win­dows, adds a shim around your main() func­tion, sets up UTF-8 ter­mi­nal en­cod­ing, en­ables AN­SI col­or es­cape codes and con­verts Uni­code com­mand-line ar­gu­ments to UTF-8 as well, en­abling you to use the same stan­dards-con­form­ing code on all plat­forms. Ad­di­tion­al­ly, it’ll al­so al­low you to hide the ter­mi­nal win­dow lurk­ing in back­ground with­out forc­ing you to im­ple­ment WinMain(). With CMake, this is all you need to do:

find_package(Corrade REQUIRED Main)
add_executable(my-application WIN32 main.cpp) # WIN32 turns it into a GUI app
target_link_libraries(my-application PRIVATE Corrade::Main)

An­oth­er thing worth men­tion­ing is that Plat­form::Sdl2Ap­pli­ca­tion and Plat­form::GlfwAp­pli­ca­tion now have ba­sic DPI aware­ness on Win­dows, catch­ing up with oth­er plat­forms.

Sin­gle-head­er li­braries

Start­ing with this re­lease, a sub­set of Mag­num func­tion­al­i­ty is be­ing ex­posed through sin­gle-head­er li­braries over at https://github.com/mosra/magnum-singles. These are all gen­er­at­ed from mul­ti-file sources and thus con­tain the best of both worlds — small foot­print of the gen­er­at­ed files as all doc­u­men­ta­tion, com­ments and non-es­sen­tial fea­tures are stripped out, but on the oth­er hand they in­her­it ex­ten­sive doc­u­men­ta­tion and >95% test cov­er­age of the orig­i­nal source code.

The li­braries were grad­u­al­ly in­tro­duced in the past posts, here’s the whole list:

Li­brary LoC PpLoC1 De­scrip­tion
Cor­radeAr­rayView.h 644 2489 Con­tain­ers::Ar­rayView and Stati­cAr­rayView, light­weight al­ter­na­tives to std::span
Cor­rade­StridedAr­rayView.h 5942 2923 Con­tain­ers::StridedAr­rayView, light­weight al­ter­na­tive to pro­posed std::mdspan.
Cor­radeAr­ray.h 6982 3344 Con­tain­ers::Ar­ray and Stati­cAr­ray, light­weight al­ter­na­tives to std::vec­tor and std::ar­ray
Cor­radeOp­tion­al.h 330 2736 Con­tain­ers::Op­tion­al, a light­weight al­ter­na­tive to std::op­tion­al
Cor­rade­Point­er.h 263 2312 Con­tain­ers::Point­er, a light­weight al­ter­na­tive to std::unique_p­tr
Cor­radeRef­er­ence.h 115 1626 Con­tain­ers::Ref­er­ence, a light­weight al­ter­na­tive to std::ref­er­ence_wrap­per
Cor­rade­Scope­Guard.h 131 34 Con­tain­ers::Scope­Guard, al­ter­na­tive to std::unique_p­tr with a cus­tom deleter
Cor­radeStl­For­war­dAr­ray.h 67 24363 For­ward dec­la­ra­tion for std::ar­ray
Cor­radeStl­For­ward­String.h 74 48 For­ward dec­la­ra­tion for std::string
Cor­radeStl­For­ward­Tu­ple.h 78 1601 For­ward dec­la­ra­tion for std::tu­ple
Cor­radeStl­For­ward­Vec­tor.h 62 7663 For­ward dec­la­ra­tion for std::vec­tor
Cor­radeStl­Math.h 57 29704 Like #include <cmath>, but with­out the heavy C++17 ad­di­tions
1.
^ lines of code af­ter a pre­proces­sor run, with sys­tem in­cludes ex­pand­ed. Gath­ered us­ing GCC 9.2 and lib­st­dc++, un­less said oth­er­wise.
2.
^ a b not a to­tal size due to in­ter-li­brary de­pen­den­cies
3.
^ a b gath­ered us­ing Clang 9.0 and libc++, since lib­st­dc++ doesn’t have a for­ward dec­la­ra­tion for std::ar­ray / std::vec­tor
4.
^ gath­ered us­ing GCC 9.2, lib­st­dc++ and -std=c++17

Test­Suite im­prove­ments, shad­er test­ing

If you’re not yet us­ing Test­Suite for tests in your Mag­num-based project (well, or any oth­er), con­sid­er giv­ing it a try. For this re­lease, con­tin­ued ef­fort was put on ren­der out­put test­ing — De­bug­Tools::Com­pareIm­age re­ceived an abil­i­ty to save a di­ag­nos­tic file in case of a com­par­i­son fail­ure, and can com­pare against an ar­bi­trary pix­el view in ad­di­tion to files and Im­ageView in­stances. Tests can be now al­so run with ver­bose out­put, show­ing de­tailed in­fo even in case the com­par­i­son pass­es:

./ShadersPhongGLTest -v --only 38
Starting Magnum::Shaders::Test::PhongGLTest with 1 test cases...
  INFO [38] renderShininess(80) at src/Magnum/Shaders/Test/PhongGLTest.cpp on line 966
        Images Containers::arrayCast<Color3ub>(_framebuffer.read(_framebuffer.viewport(), {PixelFormat::RGBA8Unorm}).pixels<Color4ub>()) and Utility::Directory::join({_testDir, "PhongTestFiles", data.expected}) have deltas 1.66667/0.0159375 below threshold 12/0.043. Delta image:
          |                                        |
          |                                        |
          |                                        |
          |       ::     :                         |
          |      Z:     :::                        |
          |    :Z        Z                         |
          |    ::   Z :  :Z:                       |
          |    Z   :Z:   Z:       :                |
          |   ::   + : : :                         |
          |     :  :  :  :     ::    :             |
          |          :               Z             |
          |        +::   :        : Z    :         |
          |   :     ++:: ZZ :   +Z::   : ::        |
          |        +ZZ: +:  :     :     Z:         |
        Top 10 out of 189 pixels above max/mean threshold:
          [19,23] #f96161, expected #fa6363 (Δ = 1.66667)
          [20,23] #ffa9a9, expected #ffabab (Δ = 1.33333)
          [21,22] #ffd9d9, expected #ffdbdb (Δ = 1.33333)
          [20,22] #ff9090, expected #ff9292 (Δ = 1.33333)
          [13,62] #250707, expected #260808 (Δ = 1)
          [28,56] #2a0808, expected #2b0909 (Δ = 1)
          [11,56] #4d0f0f, expected #4e1010 (Δ = 1)
          [31,54] #2a0808, expected #2b0909 (Δ = 1)
          [19,54] #490f0f, expected #480e0e (Δ = 1)
          [ 8,51] #6b1515, expected #6c1616 (Δ = 1)
Finished Magnum::Shaders::Test::PhongGLTest with 0 errors out of 2 checks.

With these im­prove­ments in place, the whole Shaders li­brary has tests for ren­der­ing out­put. So far, thanks to these, we ironed out a bunch of bugs in dusty cor­ner cas­es, but that’s not all — it makes fur­ther mod­i­fi­ca­tions, op­ti­miza­tions and im­prove­ments eas­i­er to make as re­gres­sions will now be caught through au­to­mat­ic test­ing.

Re­duced over­head, guar­an­teed thread safe­ty and unique­ness of glob­als

While glob­als are of­ten a source of im­mense pain, some­times hav­ing a state glob­al is the most prag­mat­ic de­ci­sion of all. Mag­num cur­rent­ly us­es glob­als in these few places:

  1. Util­i­ty::De­bug scoped out­put re­di­rect­ion and col­or­ing
  2. Each com­piled-in Util­i­ty::Re­source re­source reg­is­ters it­self in­to a glob­al stor­age
  3. Sim­i­lar­ly, stat­ic plug­ins reg­is­ter them­selves in­to Plug­in­Man­ag­er
  4. And be­cause OpenGL (and then Ope­nAL, which is mod­elled af­ter it) has a glob­al con­text, it makes sense to have cur­rent GL::Con­text / Au­dio::Con­text glob­al­ly ac­ces­si­ble as well

One oth­er us­age of glob­als was in Re­source­M­an­ag­er (and tran­si­tive­ly in De­bug­Tools::Ob­jec­tRen­der­er as well), but those APIs are now dep­re­cat­ed in fa­vor of ex­plic­it­ly passed ref­er­ences. And, for the up­com­ing Vulkan back­end, there’s no plan to have a GL-like “glob­al con­text” at all.

For this re­lease, all glob­al state was rewrit­ten to be com­plete­ly al­lo­ca­tion-free (reg­is­tra­tion of re­sources and stat­ic plug­ins is now just build­ing an in­tru­sive linked list), which means there’s no need to run any glob­al de­struc­tors for these. More­over, while al­ready very light­weight, the au­to­mat­ic reg­is­tra­tion can be com­plete­ly opt­ed out of, al­low­ing you to get rid of glob­al con­struc­tors as well.

All glob­al state that’s read-write is now made thread_local, mean­ing ev­ery thread will have its own copy of the glob­al da­ta. This makes more sense than hav­ing the glob­al state ac­cess guard­ed by a lock. Be­sides be­ing faster, you might want to re­di­rect your log out­put to a file in one thread but not in the oth­er. Apart from these, Mag­num doesn’t do any­thing about thread­ing on its own — if your app needs to share da­ta across threads, you’re ful­ly re­spon­si­ble for guard­ing against da­ta races. Thread-lo­cal vari­ables of course come with some small over­head, and if you don’t need that, you can turn it off via the COR­RADE_BUILD_­MUL­TI­THREAD­ED op­tion.

With the in­tro­duc­tion of Python bind­ings, glob­als posed an­oth­er prob­lem — if Mag­num is built stat­i­cal­ly and then linked in­to two dis­tinct Python mod­ules, the glob­als get du­pli­cat­ed, each mod­ule hav­ing its own copy. On Unix sys­tems this was eas­i­ly solved by mark­ing the few glob­als ex­port­ed weak sym­bols, telling the dy­nam­ic link­er to al­ways pick on­ly one of them. On Win­dows there’s no no­tion of a weak link­ing and ad­di­tion­al­ly __declspec(dllexport) at­tributes can’t be thread_local, so this got solved by a brown mag­ic in­volv­ing Get­P­ro­cAd­dress().

Full changel­og … and what’s next?

This re­lease took al­most 9 months to make, much more than ini­tial­ly planned, and a “re­lease cut” had to be made in or­der to keep it from grow­ing in­def­i­nite­ly. Be­cause of that, there’s a lot of things that didn’t fit in­to this an­nounce­ment and the changel­ogs are larg­er than you might ex­pect:

For the next ver­sion, apart from im­age-re­lat­ed im­prove­ments hint­ed above, well un­der­way is a re­work of Trade::Mesh­Data3D, with sup­port for more ver­tex at­tributes, ar­bi­trary da­ta types and ze­ro-copy da­ta im­port. This one will like­ly re­sult al­so in ad­di­tions to Cor­rade con­tain­er types (grow­able ar­rays) and var­i­ous oth­er things. Sub­scribe to mosra/mag­num#371 for up­dates.

Hav­ing Python bind­ings out of the way, the Vulkan bind­ings got a pri­or­i­ty as well — ex­pect Vulkan-re­lat­ed changes pop­ping up in the next months.

Up­dat­ing from pre­vi­ous ver­sions

If you’re us­ing Home­brew, MSYS pack­ages Arch­Lin­ux AUR or Vcp­kg, 2019.10 is al­ready in the repos­i­to­ries. Arch­Lin­ux com­mu­ni­ty pack­ages are sched­uled for an up­date, and Ubun­tu pack­ages can be built di­rect­ly from with­in the cloned repos­i­to­ry as usu­al.

The li­brary is con­stant­ly un­der­go­ing a “head­er hy­giene” in­clude cleanup, mean­ing you can now get com­pil­er er­rors re­lat­ed to use of in­com­plete types. The fix is in most cas­es in­clud­ing cor­re­spond­ing head­ers — in many cas­es some of these:

#include <Corrade/Containers/Reference.h>
#include <Corrade/Containers/Optional.h>
#include <Corrade/Utility/DebugStl.h>
#include <Magnum/Math/Matrix4.h>

Since it’s been over a year since the “GL split” in 2018.04, 2019.10 re­moves all com­pat­i­bil­i­ty alias­es of GL APIs in the root names­pace. If you’re up­grad­ing from old­er ver­sions, the rec­om­mend­ed way is as al­ways jump­ing over all sta­ble re­leas­es (so 2018.04, 2018.10, 2019.01) and fix­ing up what breaks, in­stead of di­rect­ly try­ing with the lat­est.

Thank you

A huge part of the work for this re­lease was done by ex­ter­nal con­trib­u­tors — sin­cere thanks to ev­ery­one (and apolo­gies to those I for­got):