May 02, 2018

Magnum 2018.04 released

with custom renderer support

The new re­lease brings more flex­i­bil­i­ty to as­set man­age­ment and ren­der­ing ab­strac­tions, im­proves plug­in han­dling and show­cas­es Leap Mo­tion in­te­gra­tion.

Two months af­ter the pre­vi­ous re­lease, Mag­num comes with the big­gest change in five years of its pub­lic his­to­ry. With 2018.04 the OpenGL wrap­ping lay­er and as­set man­age­ment APIs are no longer a core part of the li­brary, but com­plete­ly op­tion­al — this fi­nal­ly al­lows for the Vulkan port im­ple­men­ta­tion and al­so makes it pos­si­ble to use Mag­num in con­texts where it was not pos­si­ble be­fore. The changel­og since 2018.02 is al­most as long as list of changes for whole three years be­fore, so let’s dive right in!

OpenGL wrap­ping lay­er made op­tion­al

There’s a new GL li­brary con­tain­ing all buf­fer, mesh, tex­ture and oth­er APIs that were pre­vi­ous­ly part of the core li­brary. It’s built by de­fault, but can be com­plete­ly dis­abled. Some Mag­num li­braries still have a hard de­pen­den­cy on it, at the mo­ment it’s Text and Shaders. This will change dur­ing the next re­leas­es, es­pe­cial­ly the Text li­brary is sched­uled to re­ceive up­dates that sep­a­rate the text lay­out­ing from GPU ren­der­ing. How­ev­er, these up­dates will be rather in­cre­men­tal com­pared to this re­lease.

Some Mag­num li­braries pro­vide func­tion­al­i­ty that makes use of the GL li­brary (for ex­am­ple Mesh­Tools::com­pile() or De­bug­Tools::tex­ture­SubIm­age()). This doesn’t make them tied to it, how­ev­er — you can dis­able OpenGL in­te­gra­tion and leave just the API-ag­nos­tic fea­tures there. Look in the doc­u­men­ta­tion for de­tails.

API-in­de­pen­dent as­set man­age­ment

Be­sides hav­ing the GL li­brary op­tion­al, the Trade li­brary is now op­tion­al as well — so if you use Mag­num for GPU da­ta pro­cess­ing and don’t need any im­age de­cod­ing plug­ins or mesh im­port, you can use Mag­num with­out any of that, with faster build times and small­er de­ploy sizes.

To make the as­set man­age­ment APIs in­de­pen­dent on OpenGL, there are now gener­ic Pix­elFor­mat / Com­pressed­Pix­elFor­mat, Mesh­Prim­i­tive, Sam­pler­Fil­ter, Sam­pler­Mipmap and Sam­pler­Wrap­ping enums. Each of these con­tains a set of most wide­ly used for­mats/types and all im­age and scene im­porters now de­scribe da­ta us­ing those gener­ic types. The new mesh prim­i­tive and sam­pler types re­tain their orig­i­nal nam­ing, be­cause that’s in line with how most graph­ics APIs in­clud­ing Vulkan, Met­al, or D3D name them. How­ev­er I dropped the un­nec­es­sary pix­el for­mat / pix­el type dis­tinc­tion en­forced by OpenGL lega­cy and made just a sin­gle Pix­elFor­mat enum. Nam­ing of its val­ues is a re­sult of find­ing a good com­pro­mise be­tween read­abil­i­ty and clar­i­ty with un­am­bigu­ous im­plic­it map­ping to GPU APIs. It re­sem­bles MTLPix­elFor­mat the most, which it­self is a more read­able al­ter­na­tive to what Vk­For­mat or D3D for­mats of­fer.

const Color3 data[]{
    0xff0000_rgbf, 0x00ff00_rgbf,
    0x00ffff_rgbf, 0xffff00_rgbf
};

/* Three-component 32-bit floating point pixels */
ImageView2D image{PixelFormat::RGB32F, {2, 2}, data};

How­ev­er, that does not mean you are now re­strict­ed to some “low­est com­mon de­nom­i­na­tor” sub­set, as is un­for­tu­nate­ly the case in some graph­ics API ab­strac­tions — if you feel the need for a BGR565 type or sim­ply want to use the OpenGL types di­rect­ly like be­fore, just do it:

ImageView2D image{GL::PixelFormat::BGR, GL::PixelType::UnsignedShort565Rev,
    {32, 32}, data};

The Im­age, Im­ageView and Trade::Im­age­Da­ta APIs sup­port both — if you pass them a gener­ic for­mat, the GL li­brary con­verts it to the well-known GL::Pix­elFor­mat / GL::Pix­el­Type pair that OpenGL un­der­stands; if you pass it an im­ple­men­ta­tion-spe­cif­ic for­mat, it will use it as-is.

The trans­la­tion to OpenGL-spe­cif­ic for­mats is done im­plic­it­ly (i.e., all APIs in the GL li­brary that take the gener­ic enum have an over­load that takes the OpenGL-spe­cif­ic val­ue as well), but you can al­so re­quest ex­plic­it con­ver­sion us­ing GL::pix­elFor­mat(), GL::pix­el­Type(), GL::sam­pler­Wrap­ping() and oth­er trans­la­tion func­tions.

This is not lim­it­ed to OpenGL, though — if you want to use the Mag­num im­age class­es with Met­al-spe­cif­ic for­mats, for ex­am­ple, you can do so, too. In this case the con­struc­tor re­quires you to spec­i­fy pix­el size man­u­al­ly, as it needs it for var­i­ous checks and op­er­a­tions lat­er:

/* Default pixel storage, 8-bit sRGB + alpha, four bytes per pixel */
ImageView2D view{{}, MTLPixelFormatRGBA8Unorm_sRGB, {}, 4, {256, 256}, data};

Us­ing cus­tom ren­der­ers with Mag­num

Mak­ing the GL li­brary op­tion­al was pri­mar­i­ly a first step to­wards Vulkan sup­port (work on which will now be re­sumed, you can watch the progress in mosra/mag­num#234), but this opens up more pos­si­bilites than that — you can now use Mag­num in projects that have their own ren­der­ing en­gines! To prove the point, there’s now a new ex­am­ple that us­es the sokol_gfx sin­gle-file graph­ics API ab­strac­tion li­brary to ren­der a tri­an­gle on screen with­out re­ly­ing on Mag­num OpenGL sup­port at all.

sokol_gfx Triangle example screenshot
Tri­an­gle us­ing sokol_gfx
Shows how to use cus­tom ren­der­ers with Mag­num.

The pos­si­bil­i­ties don’t end here — if the bgfx li­brary is your ren­der­er of choice, you can still make use of the well-doc­u­ment­ed Mag­num Math li­brary, Scene­Graph or the OpenGex­Im­porter and oth­er plug­ins. Or use the Test­Suite with Un­re­al En­gine and com­pare your shad­er out­put against a ground truth us­ing De­bug­Tools::Com­pareIm­age, for ex­am­ple.

Plug­in work­flow im­prove­ments

A lot of ef­fort went in­to im­prov­ing the gen­er­al plug­in work­flow. Un­til now, load­ing a file us­ing plug­ins in­volved quite a lot of set­up in or­der to find and load the cor­rect plug­in from the filesys­tem. Not any­more — load­ing a mod­el from a glTF file is now just a mat­ter of this:

PluginManager::Manager<Trade::AbstractImporter> manager;
std::unique_ptr<Trade::AbstractImporter> importer =
    manager.loadAndInstantiate("GltfImporter");

importer->openFile("cave.gltf");
Containers::Optional<Trade::MeshData3D> data =
    importer->mesh3D(importer->mesh3DForName("treasure-chest"));

And the work­flow is the same in­de­pen­dent­ly on the plat­form — thanks to @Squareys the Vcp­kg pack­ages al­so re­ceived up­dates that make the plug­in “just work” on Win­dows as well, in­clud­ing de­ploy­ment of all nec­es­sary files.

There’s more — plug­in alias pri­or­i­ties, plug­in-spe­cif­ic con­fig­u­ra­tion and au­tomag­ic im­port of stat­ic plug­ins. And if you like glTF, there’s now a fresh TinyGlt­fIm­porter plug­in for im­port­ing this for­mat. Fur­ther de­tails about all plug­in us­abil­i­ty im­prove­ments are in a ded­i­cat­ed ar­ti­cle.

Pack­age up­dates

Thanks to @xypro­to, the Arch­Lin­ux [community] repos­i­to­ry now con­tains sta­ble ver­sions of Cor­rade, Mag­num and Mag­num Plug­ins pack­ages, so if you are on Arch­Lin­ux, you can just do this now:

sudo pacman -S magnum

The Home­brew pack­ages are now pro­vid­ing sta­ble ver­sions so you no longer need to in­stall --HEAD ver­sions:

brew install mosra/magnum/magnum

The Home­brew and Arch­Lin­ux AUR pack­ages are al­ready up­dat­ed to 2018.04, the Arch­Lin­ux [community] and Vcp­kg pack­ages are ex­pect­ing an up­date soon (see mi­cro­soft/vcp­kg#3407). Be­sides these, Mag­num pro­vides in-tree De­bian/ Ubun­tu and Gen­too pack­ages as usu­al, head over to the doc­u­men­ta­tion for de­tails.

More? Yes, there’s more!

Thanks to @Squareys (again!), we have a new fun ex­am­ple in­te­grat­ing the Leap Mo­tion SDK. See its full source in the doc­u­men­ta­tion.

Leap Motion example screenshot
Leap Mo­tion
(In case you won­der how can @Squareys move his hands so fast, it’s be­cause he us­es vim.)

An­droid sup­port got an over­haul, sup­port­ing the Gra­dle build and new Clang-based ND­Ks. There’s now a very de­tailed An­droid guide in the doc­u­men­ta­tion, ex­plain­ing all quirks and an­noy­ances you might en­counter. Oth­er plat­form-spe­cif­ic guides will get grad­u­al­ly filled with use­ful in­for­ma­tion re­gard­ing spe­cif­ic tar­gets.

The GL li­brary has now an ini­tial sup­port for OpenGL ES 3.2 and OpenGL 4.6 and re­ceived var­i­ous clar­i­fi­ca­tions and up­dates re­gard­ing float­ing-point for­mats on em­bed­ded sys­tems.

The Prim­i­tives names­pace was sim­pli­fied to a bunch of free func­tions in­stead of the OOP-heavy in­ter­face be­fore. There are new cone, grid and 3D cir­cle prim­i­tives and the doc­u­men­ta­tion now shows how ev­ery prim­i­tive looks:

Primitives namespace documentation

Among oth­er things the Ocu­lus VR ex­am­ple got sim­pli­fied and there’s a sec­ond ex­per­i­men­tal ver­sion of the Dart­In­te­gra­tion li­brary by @costashatz.

Up­grad­ing from pre­vi­ous ver­sions

Be­cause the API changed quite heav­i­ly, up­grad­ing from pre­vi­ous ver­sions is not as smooth as usu­al. Nev­er­the­less, great care was tak­en to make this process as fric­tion­less as pos­si­ble.

First of all, if you are not on 2018.02 yet, please up­date to it first to re­duce the amount of un­want­ed sur­pris­es. Next, in case you use Mag­num with the BUILD_DEPRECATED op­tion dis­abled (it’s usu­al­ly en­abled by de­fault), en­able it again — it will help with the up­grade a lot.

Up­date your copies of FindCorrade.cmake, FindMagnum.cmake and oth­ers — in the ide­al case just do­ing that should make your code com­pile and link again, al­though with a lot of dep­re­ca­tion warn­ings. The dep­re­ca­tion warn­ings in­clude help­ful mes­sages show­ing what to use in­stead, a high-lev­el over­view of the changes need­ed is be­low:

CMake set­up be­fore:

find_package(Magnum REQUIRED
    Sdl2Application)

add_exectutable(my-app main.cpp)
target_link_libraries(my-app PRIVATE
    Magnum::Application
    Magnum::Magnum)

With 2018.04:

find_package(Magnum REQUIRED
    GL
    Trade
    Sdl2Application)

add_exectutable(my-app main.cpp)
target_link_libraries(my-app PRIVATE
    Magnum::Application
    Magnum::GL
    Magnum::Trade
    Magnum::Magnum)

In­cludes be­fore:

#include <Magnum/ImageView.h>
#include <Magnum/PixelFormat.h>
#include <Magnum/Buffer.h>
#include <Magnum/Framebuffer.h>
#include <Magnum/Mesh.h>

With 2018.04:

#include <Magnum/ImageView.h>
#include <Magnum/PixelFormat.h>
#include <Magnum/GL/Buffer.h>
#include <Magnum/GL/Framebuffer.h>
#include <Magnum/GL/Mesh.h>

Mesh set­up be­fore:

Buffer vertices, indices;
vertices.setData(data,
    BufferUsage::StaticDraw);
indices.setData(indexData,
    BufferUsage::StaticDraw);

Mesh mesh{MeshPrimitive::Triangles};
mesh.addVertexBuffer(vertices, 0, ...)
    .setIndexBuffer(indices, 0,
        Mesh::IndexType::UnsignedShort);

With 2018.02:

GL::Buffer vertices;
vertices.setData(data,
    GL::BufferUsage::StaticDraw);
indices.setData(indexData,
    GL::BufferUsage::StaticDraw);

GL::Mesh mesh{MeshPrimitive::Triangles};
mesh.addVertexBuffer(vertices, 0, ...)
    .setIndexBuffer(indices, 0,
        MeshIndexType::UnsignedShort);

Tex­ture set­up be­fore:

ImageView2D image{
    PixelFormat::Red, PixelType::UnsignedByte,
    size, data};

Texture2D t;
t.setMagnificationFilter(
    Sampler::Filter::Linear)
 .setMinificationFilter(
    Sampler::Filter::Linear)
 .setWrapping(Sampler::Wrapping::ClampToEdge)
 .setStorage(1, TextureFormat::R8, size)
 .setSubImage(0, {}, image);

With 2018.02:

ImageView2D image{
    PixelFormat::R8Unorm,
    size, data};

GL::Texture2D t;
t.setMagnificationFilter(
    SamplerFilter::Linear)
 .setMinificationFilter(
    SamplerFilter::Linear)
 .setWrapping(SamplerWrapping::ClampToEdge)
 .setStorage(1, GL::TextureFormat::R8, size)
 .setSubImage(0, {}, image);

The above snip­pets as­sume you’d want to use the gener­ic pix­el for­mat and sam­pler state enums as it’s less typ­ing, but you can use their GL-spe­cif­ic al­ter­na­tives as well. You al­so might not need to link to all the li­braries shown above if you don’t use any func­tion­al­i­ty from them.

Look in the changel­og for a full over­view of all dep­re­cat­ed fea­tures and al­so back­wards-in­com­pat­i­ble changes.

Com­plete changel­og

There’s a lot of small changes that would be too bor­ing to men­tion in full here. As usu­al, you can find a com­plete list of changes since 2018.02 in the doc­u­men­ta­tion:

Spe­cial thanks

Again a lot of work in this re­lease is done thanks to ex­ter­nal con­trib­u­tors:

  • To­bias Stein (@Nussknack­erXXL) — ini­tial im­ple­men­ta­tion of the TinyGlt­fIm­porter plug­in
  • Jonathan Hale (@Squareys) — Ocu­lus VR ex­am­ple sim­pli­fi­ca­tion, con­tri­bu­tions to the tiny_gltf im­porter plug­in, Leap Mo­tion ex­am­ple, Vcp­kg pack­age main­te­nance and much more
  • Kon­stan­ti­nos Chatzi­lyger­oud­is (@costashatz) — im­prove­ments to the DART in­te­gra­tion li­brary
  • Alex­an­der F Rød­seth (@xypro­to) — Arch­Lin­ux [community] pack­age main­te­nance

Thanks a lot to ev­ery­one, not to for­get all peo­ple that re­port­ed is­sues or just wrote en­cour­ag­ing mes­sages on the Git­ter chat. Cheers!