May 2015 updates to Magnum

The Mag­num C++11/C++14 and OpenGL graph­ics en­gine gained OpenGEX format sup­port, a bunch of new im­port­er plu­gins, cross-plat­form ex­ten­sion load­er, trans­form feed­back sup­port and new fea­tures from OpenGL ES 3.1 and OpenGL 4.5, among oth­er things.

The new re­lease comes al­most a whole year after the pre­vi­ous one, mainly due to the fact that I’m cur­rently full-time em­ployed and I have less free time than I ex­pec­ted. Nev­er­the­less the pro­ject ac­cu­mu­lated quite a lot of changes over the year, so this changelog may be a bit over­whelm­ing. I plan to do one smal­ler re­lease in a few months to pol­ish and clean up some more things and then I’ll prob­ably start break­ing stuff in nev­er-be­fore-seen ways to make Vulkan sup­port a real­ity.

The ver­sion 2015.05 is avail­able un­der the v2015.05 tag in Cor­rade, Mag­num, Mag­num Plu­gins and Mag­num In­teg­ra­tion Git­Hub re­pos­it­or­ies. This art­icle will de­scribe the most im­port­ant changes, for de­tailed list fol­low the changelog links at the end of this an­nounce­ment.

For com­pat­ib­il­ity branch there is the snapshot-2015-05-compatibility tag in Cor­rade, Mag­num, Mag­num Plu­gins and Mag­num Ex­amples re­pos­it­or­ies.

Plu­gin ali­ases, smart im­port­ers and new format sup­port

In an on­go­ing at­tempt to make the en­gine use­ful even without a slew of ex­tern­al lib­rary de­pend­en­cies, I ad­ded a bunch of de­pend­ency-less plu­gins for load­ing com­mon scene and im­age formats (see the changelog be­low for a com­plete list). I had a firsthand ex­per­i­ence about how crazy hard it is to get work­ing Free­type, Harf­Buzz or even libPNG bin­ary for Win­dows, so there are more plu­gins planned (es­pe­cially for de­pend­ency-less font hand­ling), stay tuned.

In ad­di­tion to this, the plu­gin load­er now sup­ports ali­ases. It means that, to load PNG im­ages, for ex­ample, you no longer need to worry about which par­tic­u­lar plu­gin might be avail­able on giv­en plat­form and just al­ways re­quest PngIm­port­er plu­gin:

PluginManager::Manager<Trade::AbstractImporter> manager{MAGNUM_PLUGINS_IMPORTER_DIR};
std::unique_ptr<Trade::AbstractImporter> importer = manager.loadAndInstantiate("PngImporter");
// ...

Now, if the PngImporter plu­gin is not avail­able, it tries to load any oth­er plu­gin that ali­ases with PngImporter. Cur­rently it might be for ex­ample StbIm­ageIm­port­er, in fu­ture re­leases pos­sibly also Sdl2ImageImporter or WebGlImageImporter.

Lastly, there are now smart im­age and scene load­ers, named Any­Im­ageIm­port­er and Any­SceneIm­port­er. They try to de­tect file type from fi­le­name and then re­dir­ect the op­er­a­tion to ded­ic­ated im­port­er plu­gin (so if you load image.tga, for ex­ample, it tries to load it with TgaImporter, if you load texture.png, it tries to load it with PngImporter etc.). In com­bin­a­tion with plu­gin ali­ases this makes a fairly power­ful im­port­er frame­work.

Ini­tial OpenDDL/OpenGEX sup­port

Pos­sibly the biggest new im­port­er plu­gin is the OpenGEX format im­port­er. If you haven’t heard about it yet, it is a fairly new 3D scene file format based on the cleanly de­signed OpenDDL lan­guage. It aims to be a bet­ter al­tern­at­ive to the bloated XML-based COL­LADA format and cur­rently there are ex­port­ers from 3Ds Max, Maya and Blender.

Cur­rent OpenGEX im­port­er im­ple­ment­a­tion in Mag­num cov­ers the same fea­ture set as the ColladaImporter ob­sol­ete plu­gin, but with faster and more ro­bust im­ple­ment­a­tion and hope­fully with less bugs and un­covered corner cases. The fu­ture of the COL­LADA im­port­er is un­cer­tain and it will prob­ably be de­prec­ated in fa­vor of the new OpenGEX im­port­er, be­cause COL­LADA sup­port in 3D mod­el­ling soft­ware is far from ideal and the XML pars­er is hell to im­ple­ment.

The un­der­ly­ing OpenDDL pars­er that powers the plu­gin is cur­rently only a private part of the plu­gin, but if it is seen to be use­ful out­side of the plu­gin, I may move it in­to a pub­licly us­able lib­rary. Here’s an ex­ample how OpenDDL file pars­ing can look like:

OpenDdl::Document d;

/* Parse the document */
if(!d.parse(data, OpenGex::structures, OpenGex::properties)) {
    // ...
}

/* Validate its structure */
if(!d.validate(OpenGex::rootStructures, OpenGex::structureInfo)) {
    // ...
}

/* Parse all meshes */
for(OpenDdl::Structure geometryObject: d.childrenOf(OpenGex::GeometryObject)) {
    /* Decide about primitive */
    if(std::optional<OpenDdl::Property> primitive = geometryObject.findPropertyOf(OpenGex::primitive)) {
        auto&& str = primitive->as<std::string>();
        if(str == "triangles") {
            // ...
        } else if(str == "lines") {
            // ...
        } // ...
    } else {
        // default primitive ...
    }

    /* Parse vertex array */
    OpenDdl::Structure vertexArray = geometryObject.firstChildOf(OpenGex::VertexArray);
    auto&& attrib = vertexArray.propertyOf(OpenGex::attrib).as<std::string>();
    if(attrib == "position") {
        // ...
    } else if(attrib == "normal") {
        // ...
    }

    /* Parse vertex array data */
    Containers::ArrayReference<const Float> vertexData = vertexArray.firstChild().asArray<Float>();
    // ...
}

The View­er ex­ample was also up­dated to sup­port load­ing OpenGEX files and it is now also avail­able on­line through Em­scripten:

Viewer example screenshot
View­er Ex­ample webgl1
Uses scene im­port­er plu­gins to load a 3D scene for in­ter­act­ive view­ing.

New ex­ten­sion load­er, ex­ten­sion load­ing for OpenGL ES

I was be­ing more and more dis­sat­is­fied with the state of glLoadGen (which was ori­gin­ally meant to re­place the even worse GLEW) and switched to flex­t­GL. If you don’t know it yet, it is very flex­ible and cus­tom­iz­able ex­ten­sion load­er gen­er­at­or and con­sists of just a single very clean Py­thon file. Switch­ing to it res­ul­ted in much short­er gen­er­ated code (which means faster build times) and after a few minor changes I man­aged to add OpenGL ES sup­port to it. So it is now fi­nally pos­sible to use ex­ten­sions on both desktop GL and OpenGL ES. The only plat­forms which don’t have any no­tion of tra­di­tion­al ex­ten­sion load­ing are WebGL and NaCl, on which the ex­ten­sion hand­ling is not yet fully im­ple­men­ted.

OpenGL 4.5, OpenGL ES 3.1 sup­port

In Au­gust 2014, OpenGL 4.5 was re­leased, which was un­der­stood as some sort of a cleanup re­lease with no rad­ic­ally new func­tion­al­ity. The most es­sen­tial ad­di­tion is AR­B_­dir­ec­t_state_ac­cess, which fi­nally cleans up some rough corners of the old EX­T_­dir­ec­t_state_ac­cess ex­ten­sion and puts it in the core spe­cific­a­tion. Mag­num cur­rently im­ple­ments both ex­ten­sions and prefers the ARB one. Some API changes were needed to ac­co­mod­ate to the new design, mainly in con­struc­tion of query ob­jects and cube map tex­ture data up­loads. The oth­er minor fea­tures in­clude abil­ity to query only a part of tex­ture im­age, in­ver­ted con­di­tion­al ren­der­ing etc., see be­low for com­plete list.

Thanks to the new ex­ten­sion load­er it was also fi­nally pos­sible to im­ple­ment OpenGL ES 3.1 sup­port. Un­like the ES 2.0/3.0 switch, there is no com­pile-time op­tion to en­able OpenGL ES 3.1, the lib­rary is com­piled for both 3.0 and 3.1 and the new fea­tures are en­abled based on ver­sion re­por­ted by the driver.

IRC chan­nel

In ad­di­tion to all oth­er com­mu­nic­a­tion chan­nels there is now a IRC chan­nel: join the dis­cus­sion at #magnum-engine on Freen­ode.

Com­plete changelog

You can find a de­tailed list of changes in ver­sion 2015.05 in the doc­u­ment­a­tion: