May 2015 updates to Magnum

The Mag­num C++11/C++14 and OpenGL graph­ics en­gine gained OpenGEX for­mat sup­port, a bunch of new im­porter plug­ins, cross-plat­form ex­ten­sion load­er, trans­form feed­back sup­port and new fea­tures from OpenGL ES 3.1 and OpenGL 4.5, among oth­er things.

The new re­lease comes al­most a whole year af­ter the pre­vi­ous one, main­ly due to the fact that I’m cur­rent­ly full-time em­ployed and I have less free time than I ex­pect­ed. Nev­er­the­less the project ac­cu­mu­lat­ed quite a lot of changes over the year, so this changel­og may be a bit over­whelm­ing. I plan to do one small­er re­lease in a few months to pol­ish and clean up some more things and then I’ll prob­a­bly start break­ing stuff in nev­er-be­fore-seen ways to make Vulkan sup­port a re­al­i­ty.

The ver­sion 2015.05 is avail­able un­der the v2015.05 tag in Cor­rade, Mag­num, Mag­num Plug­ins and Mag­num In­te­gra­tion GitHub repos­i­to­ries. This ar­ti­cle will de­scribe the most im­por­tant changes, for de­tailed list fol­low the changel­og links at the end of this an­nounce­ment.

For com­pat­i­bil­i­ty branch there is the snapshot-2015-05-compatibility tag in Cor­rade, Mag­num, Mag­num Plug­ins and Mag­num Ex­am­ples repos­i­to­ries.

Plug­in alias­es, smart im­porters and new for­mat sup­port

In an on­go­ing at­tempt to make the en­gine use­ful even with­out a slew of ex­ter­nal li­brary de­pen­den­cies, I added a bunch of de­pen­den­cy-less plug­ins for load­ing com­mon scene and im­age for­mats (see the changel­og be­low for a com­plete list). I had a first­hand ex­pe­ri­ence about how crazy hard it is to get work­ing Freetype, Harf­Buzz or even libP­NG bi­na­ry for Win­dows, so there are more plug­ins planned (es­pe­cial­ly for de­pen­den­cy-less font han­dling), stay tuned.

In ad­di­tion to this, the plug­in load­er now sup­ports alias­es. It means that, to load PNG im­ages, for ex­am­ple, you no longer need to wor­ry about which par­tic­u­lar plug­in might be avail­able on giv­en plat­form and just al­ways re­quest PngIm­porter plug­in:

PluginManager::Manager<Trade::AbstractImporter> manager{MAGNUM_PLUGINS_IMPORTER_DIR};
std::unique_ptr<Trade::AbstractImporter> importer = manager.loadAndInstantiate("PngImporter");
// ...

Now, if the PngImporter plug­in is not avail­able, it tries to load any oth­er plug­in that alias­es with PngImporter. Cur­rent­ly it might be for ex­am­ple StbIm­ageIm­porter, in fu­ture re­leas­es pos­si­bly al­so Sdl2ImageImporter or WebGlImageImporter.

Last­ly, there are now smart im­age and scene load­ers, named Any­Im­ageIm­porter and AnySceneIm­porter. They try to de­tect file type from file­name and then re­di­rect the op­er­a­tion to ded­i­cat­ed im­porter plug­in (so if you load image.tga, for ex­am­ple, it tries to load it with TgaImporter, if you load texture.png, it tries to load it with PngImporter etc.). In com­bi­na­tion with plug­in alias­es this makes a fair­ly pow­er­ful im­porter frame­work.

Ini­tial OpenD­DL/OpenGEX sup­port

Pos­si­bly the big­gest new im­porter plug­in is the OpenGEX for­mat im­porter. If you haven’t heard about it yet, it is a fair­ly new 3D scene file for­mat based on the clean­ly de­signed OpenD­DL lan­guage. It aims to be a bet­ter al­ter­na­tive to the bloat­ed XML-based COL­LA­DA for­mat and cur­rent­ly there are ex­porters from 3Ds Max, Maya and Blender.

Cur­rent OpenGEX im­porter im­ple­men­ta­tion in Mag­num cov­ers the same fea­ture set as the Col­ladaIm­porter plug­in, but with faster and more ro­bust im­ple­men­ta­tion and hope­ful­ly with less bugs and un­cov­ered cor­ner cas­es. The fu­ture of the COL­LA­DA im­porter is un­cer­tain and it will prob­a­bly be dep­re­cat­ed in fa­vor of the new OpenGEX im­porter, be­cause COL­LA­DA sup­port in 3D mod­el­ling soft­ware is far from ide­al and the XML pars­er is hell to im­ple­ment.

The un­der­ly­ing OpenD­DL pars­er that pow­ers the plug­in is cur­rent­ly on­ly a pri­vate part of the plug­in, but if it is seen to be use­ful out­side of the plug­in, I may move it in­to a pub­licly us­able li­brary. Here’s an ex­am­ple how OpenD­DL file pars­ing can look like:

OpenDdl::Document d;

/* Parse the document */
if(!d.parse(data, OpenGex::structures, OpenGex::properties)) {
    // ...
}

/* Validate its structure */
if(!d.validate(OpenGex::rootStructures, OpenGex::structureInfo)) {
    // ...
}

/* Parse all meshes */
for(OpenDdl::Structure geometryObject: d.childrenOf(OpenGex::GeometryObject)) {
    /* Decide about primitive */
    if(std::optional<OpenDdl::Property> primitive = geometryObject.findPropertyOf(OpenGex::primitive)) {
        auto&& str = primitive->as<std::string>();
        if(str == "triangles") {
            // ...
        } else if(str == "lines") {
            // ...
        } // ...
    } else {
        // default primitive ...
    }

    /* Parse vertex array */
    OpenDdl::Structure vertexArray = geometryObject.firstChildOf(OpenGex::VertexArray);
    auto&& attrib = vertexArray.propertyOf(OpenGex::attrib).as<std::string>();
    if(attrib == "position") {
        // ...
    } else if(attrib == "normal") {
        // ...
    }

    /* Parse vertex array data */
    Containers::ArrayReference<const Float> vertexData = vertexArray.firstChild().asArray<Float>();
    // ...
}

The View­er ex­am­ple was al­so up­dat­ed to sup­port load­ing OpenGEX files and it is now al­so avail­able on­line through Em­scripten:

Viewer example screenshot
View­er Ex­am­ple wasm we­bgl1
Us­es scene im­porter plug­ins to load a 3D scene for in­ter­ac­tive view­ing.

New ex­ten­sion load­er, ex­ten­sion load­ing for OpenGL ES

I was be­ing more and more dis­sat­is­fied with the state of glLoadGen (which was orig­i­nal­ly meant to re­place the even worse GLEW) and switched to flextGL. If you don’t know it yet, it is very flex­i­ble and cus­tom­iz­a­ble ex­ten­sion load­er gen­er­a­tor and con­sists of just a sin­gle very clean Python file. Switch­ing to it re­sult­ed in much short­er gen­er­at­ed code (which means faster build times) and af­ter a few mi­nor changes I man­aged to add OpenGL ES sup­port to it. So it is now fi­nal­ly pos­si­ble to use ex­ten­sions on both desk­top GL and OpenGL ES. The on­ly plat­forms which don’t have any no­tion of tra­di­tion­al ex­ten­sion load­ing are We­bGL and Na­Cl, on which the ex­ten­sion han­dling is not yet ful­ly im­ple­ment­ed.

OpenGL 4.5, OpenGL ES 3.1 sup­port

In Au­gust 2014, OpenGL 4.5 was re­leased, which was un­der­stood as some sort of a cleanup re­lease with no rad­i­cal­ly new func­tion­al­i­ty. The most es­sen­tial ad­di­tion is AR­B_­di­rec­t_s­tate_ac­cess, which fi­nal­ly cleans up some rough cor­ners of the old EX­T_­di­rec­t_s­tate_ac­cess ex­ten­sion and puts it in the core spec­i­fi­ca­tion. Mag­num cur­rent­ly im­ple­ments both ex­ten­sions and prefers the ARB one. Some API changes were need­ed to ac­co­mo­date to the new de­sign, main­ly in con­struc­tion of query ob­jects and cube map tex­ture da­ta up­loads. The oth­er mi­nor fea­tures in­clude abil­i­ty to query on­ly a part of tex­ture im­age, in­vert­ed con­di­tion­al ren­der­ing etc., see be­low for com­plete list.

Thanks to the new ex­ten­sion load­er it was al­so fi­nal­ly pos­si­ble to im­ple­ment OpenGL ES 3.1 sup­port. Un­like the ES 2.0/3.0 switch, there is no com­pile-time op­tion to en­able OpenGL ES 3.1, the li­brary is com­piled for both 3.0 and 3.1 and the new fea­tures are en­abled based on ver­sion re­port­ed by the driv­er.

IRC chan­nel

In ad­di­tion to all oth­er com­mu­ni­ca­tion chan­nels there is now a IRC chan­nel: join the dis­cus­sion at #magnum-engine on Freen­ode.

Com­plete changel­og

You can find a de­tailed list of changes in ver­sion 2015.05 in the doc­u­men­ta­tion: