digital home of MacSlow

Good lord, are you really that afraid Apple? Check this this bug-entry.

This works now on nvidia-based systems and should also run on machines with i945/i950/i965-based graphics. Many color-conversions still missing though. GPU-Filters not hooked up yet.

Still not push-worthy is my messing around with emboss/blur/sobel-effects done via the GPU. While fun and easy to implement on nvidia, doing the same on the i915 is harder, due to its limits in the GL-driver. The lack of pbuffers or FBOs for the i915 ruins much of the fun. I use a workaround via glCopyTexImage2D(), but that’s nasty and is taxing the CPU/bus. What I gain in moving the filters onto the GPU, gets lost again by doing glCopyTexImage2D(). That’s needed for stacking filters on top of each other or separating the 2D-gaussian-blur in two 1D-blur-passes in order to speed up this heavy convolution filter. Furthermore a 11×11 gaussian kernel is the best I could shoe-horn on the i915 GPU.

I care about:
Yesterday was my last day at Fluendo. I have finished my internship and will return to Germany next week. In between is GUADEC’07, which I will attend. There I will give two talks. My second GUADEC and I give talks… not just one, but two! It feels so unreal… but rock&roll!

During my internship I learned a lot (not necessarily about gstreamer, as one might expect *g*) and certainly grew due to this experience. Also I believe to have left my mark upon Fluendo. This sounds bold and maybe arrogant I can imagine, but is true to some extend. All in all it will benefit the free desktop in the long run. That is my sincere hope at least!

The time I chose to come to Barcelona for my internship turned out to be very eventful. In general in the IT-world, at Fluendo and for me personally. For example the result of the Ubuntu/Dell deal went online and turned into reality. That’s a major turning point in my opinion. Fluendo lost some of their - in my opinion - most valuable employees. That’s very sad! Myself, I had a couple of very interesting interviews, phone-calls and chats with numerous people, which make my head spin in excitement. Moreover I was invited to become member of a judging jury for OpenSource-projects at an event in Portugal… I mean come one… how flattering is that?

you might care about:
Carl Worth is going after bottlenecks and non-optimized paths in EXA puts a broad smile of anticipation on my face.

In recent times I had lowfat running on a Samsung Q1 Ultra (photo of device from flickr) and a ThinkPad X60 (photo of device from flickr). Sadly these machines are just single-touch and where not mine. Still these short periods of access to this type of hardware clearly showed me that I could already do very nice stuff with just single-touch, if I would have permanent access to this kind of hardware. I have to find a way to grab hold of one of those newer X61 tablets and a Q1 Ultra! Not having this for working on lowfat makes me feel I’m missing the future.

There were some first MultiTouch-efforts from some folks over at Denmark using lowfat. Super sweet stuff! (Sorry Jens, to not have blogged about this earlier!) *sigh* I still need to build my own multi-touch setup.

Having fun with my new gstreamer/OpenGL-fu.

(click to play back, ogg/theora, ~2.4 MBytes)

The code for gl-gst-player I uploaded to f.d.o in my cozy git-repo here. It is unfinished, unpolished, undocumented and probably will not work on anything but i915 or i945 graphics hardware (only tested on Ubuntu 7.04 and Fedora 7 sofar). There are tons of remaining issues, missing features and other things I want to do with. Be aware of that before you grab it. After all, this is my gstreamer-playground for lowfat so never expect gl-gst-player to ever be something that can be considered finished.

own deeds:
After the last post regarding my OpenGL/gstreamer video-player I was contacted by a few people asking me for the source. I refused these requests sofar… almost. It was in a sorry state and still isn’t what I would be proud of, although it is better now. Nevertheless some people from Tandberg asked for it, in order to demonstrate the stylish power of mixing OpenGL with gstreamer for video-conferencing applications… and to add some extra bling-punch to a presentation they needed to give for their managers. So I gave in and send them the tarball. They hooked my code up with their webcam stuff (some of their equipment is able to capture 720p at 60Hz 30Hz!) and had a pretty successful presentation. The result of this you see in the left picture (this is using a more modest setting of 640×480@30Hz). I hope they will obey the LGPL I put my code under. Just received their tarball. I will see to sort and clean things as good as I can and upload the stuff this coming weekend at my cozy f.d.o spot.

(click to view fullsize)

(click to view fullsize)

The right picture shows the current state of affairs (playing back one of Microsoft’s WMV9/HD-example videos, which are publicly available *g*). gl-gst-player is now able to play back any video I can throw at it, e.g. ogg/Dirac (yes Dirac using the Schroedinger implementation! I tested that with a dirac-encoded version of “Elephants Dream”), ogg/Theora, mov/H.264, wmv/VC1 and all the other things gstreamer can digest. By now I was able to make some progress and have fragment-shader-based (only using the ARB-extensions, 4 multi-texture units, one texture for the yuv-frame and one for the mask-texture) YUV2RGB colorconversion (I420 to RGB to be more precise) running on my i915. This is on the application-side not at the driver-level. Now I can even watch HD-video on my Vaio which e.g. totem cannot play back smoothly or at all (mainly due to XV failing). Before starting to polish the code and put it in my public git-repository at f.d.o I want to finish the other filter-effects I need (gloom and a special blur). Sadly I don’t have FBOs or pbuffers available on my i915 and am restricted to glCopyTexSubImage2D()… and I wish I had GLSL available for the i915 *sigh* If all goes well and I find the needed time to do all that, this might still happen before GUADEC.

deeds at Fluendo:
Here is a nice example of the kind of the abstraction level Python-developers feel right at home at
The blue thing (PowerBook) controlls the green thing (Nokia cell-phone) to issue commands within a python-session triggering bluetooth-commands, that are sent to the red thing (Vaio) acutally controlling the current upstream version of elisa. In this particular case, which is captured on the photo, is it selecting/playing/pausing a video. By now - two days later after the photo was taken - the bluetooth input-provider in elisa works good enough to just use the cell-phone (currently only tested on one special Nokia phone) as a bluetooth-remote for elisa running on a bluetooth-enabled computer.

(click to view fullsize)

This is the deed of Arek and Florian, two employees at Fluendo, working late at night on the weekend… abusing Arek’s PowerBook, Florians cell-phone and my Vaio. Too bad my own cell-phone isn’t bluetooth-capable and lacks a built-in python interpreter. BTW, on Florian’s cell-phone you can even write OpenGL|ES programs via python. That’s seriously nifty!

(click to view fullsize)

A small video-player I always wanted to write myself… using - to different extends - gstreamer, OpenGL, gtk+ and cairo. The use of gstreamer and OpenGL is obvious, if you look at the screenshot. gtk+ is merely used to provide the typical event-handling and windowing-system glue-code (gtkglext in this case) and cairo is used for the masking out of the edges and getting the rounded corners. It uses multi-texturing and fragment-shading. The remaining things to implement are: using fragment-shaders for the colorspace-conversion too, hooking up some implicit-animation love for switching between different videos. Once that’s in place I would like to see some of this move into totem (if possible) to beautify transitions while switching from one video to another in case you’ve several videos in your playlist.

EDIT: This is running on an i915 on pure FOSS-drivers under metacity. It would even work under compiz using the same system… if the i915-driver would not have the known issues with OpenGL under a composited environment. On a GPU from nvidia - using their proprietary driver - it would work under compiz of course.

Already the first week of my internship at Fluendo in Barcelona has passed. Time flies here currently… so much to learn and do. Fluendo and all the people working here are a really nice bunch of folks. I’m enjoying my time here!

The last few days I’ve been starting to learn Python and the pigment-API.

There’s some very nice stuff ahead for Elisa and pigment. It’s going to be a fun ride! Elisa has got a new website, check it out. Here’s a screencast of my first ever Python-program (also using pigment for the first time):

(click to play back, ogg/theora, ~3.9 MBytes)

While it is not meant to serve any real-world purpose, it is a mere vehicle of learning for me. Before I start with the real tasks (e.g. getting cairo/pango-utilization into pigment) this messing around is needed to understand features and limits of pigment.

For any questions either use the comments here, email me or join the channels #pigment and #elisa on Freenode.

Yesterday I went to the cinema with Thomas, Christian and David to see 300. A bloody movie… literally, but still good entertainment. Though they could have de-emphasize the decapitations-scenes… or the dismemberment some

People have asked me for the slides of my talk. Note, that I generally don’t see much sense in putting just the slides of a talk online. In general a video-recording of a talk would make more sense. But this is not available, as there was no recording done in the GNOME dev-room at FOSDEM 2007. Be sure to read the license information once you grabbed the tarball of the talk from here. From the audience reactions I got during the talk and after it, I conclude that it was well received and people enjoyed it (and hopefully learnt something they did not know yet). I was really very nervous especially since some of the maintainers and contributors of projects I stressed and “criticized” sat in the audience. In retrospect, considering what I got to know about X.org and cairo at this FOSDEM-weekend, my talk should have had a revamp before presenting it. But this flexibility-presentation-fu I do not master yet. Keith Packard clearly put this into one sentence going something like: “You have to live with these kind of things, when dealing with the bleeding edge.”. Who am I to argue with the Packard *g*

FOSDEM itself was great. I almost managed to meet all the people I planned to meet and talk with them about the things that are on my mind. Sadly the brainstorming I wanted to do with Sven and Philip totally did not take place. I blame myself for not trying hard enough and being too distracted by the amount of other interesting things at FOSDEM. But at least Sven and Tim made sure I remember to stick around the gtk-devel mailing-list for discussing the things I wish would be available in future versions of gtk+. BTW, did you know that gtk+ and X.org need contributing man-power? Step in, contribute now and become world-famous!

As somehow expected, I mainly hang around with the X.org and gstreamer people as those projects are among the most interesting ones for me… from a desktop-graphics-stack perspective. I was glad to be able to meet Carl Worth, Keith Packard (the stuff X.org is made of), Michel Dänzer again and was lucky to meet Kristian Høgsberg (his talk on AIGLX… lots of very crucial information) and Øyvind Kolås (check out his talk on GEGL… I had no idea!) in person for the first time. Furthermore I also met Stéphane Marchesin again this year and enjoyed his talk about “X.org Myths”. Kristian’s call to arms “Let’s finish COMPOSITE” and Matthias thorough lecture titled “Video on dope” were also great. Sadly I didn’t make it in time for Stéphane’s update on nouveau and Keith’s summary on video drivers.

I also could not make it to Edward’s talk about “What’s new in gstreamer” and I could barely make it to Miguel’s Mono workshop. FOSDEM was really packed with a lot of things worth seeing… too much to see it all. The schedule made choosing talks to attend very hard for me this year.

I had good chunks of time chatting with Thomas and Philippe about Elisa. I’m looking forward to my time at Fluendo in a few weeks. As part of my requirements for an degree in computer science I need to do an internship. Finally the talks with Christian at Vilanova last year turn out to become reality. I’m thankful for this opportunity! My semester abroad, also needed for the degree, is hopefully going to turn into reality in 2008 as well. Then I aim for Bosten, USA.

I was pleasantly surprised to see Nat Friedman attending FOSDEM and being able to meet and talk with him in person for the first time ever. Also a great opportunity was meeting Michael Meeks, right after I gave my talk. I think Michael was around GUADEC’06 and LRL’06, but only this FOSDEM-weekend it worked out to be possible to chat with him a bit. A modest attempt to file a bug with OpenOffice in real life failed. So here is the result of doing things the correct way *g*

A large portion of people I saw again this time around are not mentioned here. Nevertheless meeting up with them again was as enjoyable as meeting folks I just mentioned. But I draw the line here as I don’t want to bore readers to death *g*

Finally, thanks to Thomas for his great guidance to the restaurant landscape in Brussels this weekend, and thanks to intel for paying for the great dinner on sunday evening.

Oh, there is one more thing… within the next 24h lowfat will start its breathing in the open at f.d.o, but I still have to do some preparations. It’s tough for me as the amount of and work (and noise) I expect is going to be huge. It will get a dedicated blog-entry.

I was sponsored to attend the UDS 2006. Holy s**t! What a week! What a intense time! What a cool community-bunch! What an assorted amount of great opportunities! First of all huge thanks to Mark Shuttleworth, Claire Newman, Daniel Holbach and Jono Bacon for each playing a part in getting me over there! I’m still stunned by this very fact. The chance to meet and talk with people like Matt Zimmerman, Rodrigo Parra Novo, Brandon Holtsclaw, Scott James Remnant and other core people from the Ubuntu-community and Canonical for the first time was terrific. It was nice to get a first hand experience of how a distribution is put together and developed step by step at the planning level. There’s really a ton of hard work involved in this process. Another nice thing to experience was the easy getting along of e.g. upstream Gnome and KDE people. Nobody questioned the other ones preferences. This is a good example to follow!

The UDS 2006 was hosted by Google (building 44, former SGI building, Mountain View, California). Google’s geek-herder par excellence, Leslie Hawthorn, made sure everything ran smoothly on the Google-side of things. She’s also the one running the “Summer Of Code”-program at Google. Thus she has some wicked geek-herding experience. Google is a very nice place to work at. Totally awesome food hands down! Non-compromise infrastructure and conferencing equipment all over the place (VoIP, video-converencing, whiteboards, projectors… you name it). The whole surroundings there felt very inspiring. Another nice fact to boot was that, besides all UDS-participants, most of the google employees run Linux exclusively on their workstations (everybody there seems to have at least two 24″ flat-panels connected to their machines) too. That’s quite something to experience for somebody like me coming from an environment where I’m the only OpenSource/Linux person in a sea of Windows-users. The only drawback there was the non-reliable wifi-hotspot. Having roughly 100 folks on one wlan-router isn’t a good idea.

Some obligatory name-dropping… I met again with Sebastien Bacher, Raphael Slinckx, Mike Hearn, Christian Schaller, Wim Taymans, Alex Graveley, Christian Kellner, Murray Cumming, Jonathan Ridell. The cozy mood of gathering around a campfire with “old-time” friends was also in effect again. First time meeting with Tim Müller, David Schleef, Jorge Castro, Melissa Draper, Evan D’Andrea, Joey Stanford, Dennis Kasprzyk, Jason Smith, Colin Watson, Ryan Lortie, Lennart Poettering and Simon McVittie and lot of other people I whose names I cannot fully remember.

A big thanks to everybody for the very funky week I had in Silicon Valley!

I took the opportunity to head to San Jose (just 15 miles away from Mountain View) during the week of the summit, because nVIDIA held the GeForceLAN 3 there. While the whole LAN-gaming is not my thing the scheduled world-wide launch of the G80 (now known as GeForce 8800 GTX or GTS) interested me far more. It was great to attend such a launch-event. I won’t list all the chips features (you can read about them elsewhere), but let me tell you… it’s a nice pixel- and vertex-pushing beast! BTW, all but one of their four tech-demos (cascade, frogger, smoke in a box, virtual Adrianne Curry) they showed on stage were running on OpenGL. Only cascade was not using OpenGL. So much for DirectX10 *g*. Aside from the launch-event itself I was able to chat a bit with the very few technical-staffers from nVIDIA and even the man himself… Jen-Sun Huang. I tried my best to do a fair bit of guerilla-marketing for OpenSource, Linux desktop-graphics and Ubuntu by running around the event floor chatting with people and answering question of people, who recognized the Ubuntu-shirt I was wearing. But this was not planned and only happened by chance. Nevertheless it had left a paramount positive impression with showing them all the “big guns” (gnome, beagle, compiz/beryl, lowfat, deskbar, my other desktop-graphics hacks etc.) I had installed on the laptop I carried around with me. You certainly get them with the eye-candy bling *g*

nVIDIA had two of their Linux-engineers sent over to the summit on thursday. Thus I was able to chat a bit with Andy Ritger and James Jones during the X11-related sessions. Nice things coming our way in terms of OpenGL 3.0 soon. Apart from that the new geometry-shaders and the overall unified shader-architecture of the new G80-chip is exposed in the Linux-drivers right from the start. It’s actually totally transparent to application developers using GLSL. Just in case you were wondering. Oh, btw… the nVIDIA folks know about the black-windows issue when running a composited desktop and your card is running out of texture-memory. They are working on it. Furthermore I hinted them towards us OpenSource folks wanting to see nVIDIA’s PureVideo HD chip-features exposed in the Linux-drivers too. They have plans for that, but not in the immediate future. I tried to hook them up with the gstreamer people.

Here is the bunch of photos I was able to make during the week.

This just came in…

Attilio Drei compiled a stock 0.3.2 version of cairo-clock under MacOS X with the new gtk+-port to quartz. Except for some themes and the window-titlebars the bling from Linux works on MacOS X too. A good indication that the work Imendio did sofar on the gtk-quartz project is solid. Thumbs up! I wonder if the shaped input-masks work too.

This is going to be a very long entry… I promise you… and I wonder how much flak I’ll get for this.

The recent months have clearly demonstrated that the average user of an OpenSource-system enjoys a healthy amount of bling. Just every computer-user is “guilty” of this. Some to a larger degree, some to a lesser one. The current state of things looks actually pretty good on our side of the fence… at least as long as inter-window effects are concerned (think: all effects made available via compositing-managers like compiz). Several effects are really improving usability or general accessibility (e.g. cube, expose-like scale, tab-switcher, unfold, desktop-zoom) and then there are some just for fun (e.g. wobbly, skydome). But sadly it stops here.

Those afore mentioned “inter-window” effects are just one part of the equation. The other paramount portion is made up of application-level or UI-toolkit-level effects. No idea what I mean? Some (admittedly) extreme examples:

• pulsing buttons
• smooth fading transitions or animated movement hiding or unhiding widgets or areas
• smoothly scaling up or down widgets or other UI-toolkit-level elements
• what about “cleanly animate and flip-rotate these video-texture surfaces from here to there”
• a consistent appeal and look between inter-window- and toolkit-level effects
• seamless integration of realtime-3D, 2D-vector-graphics, video and UI-elements in one application
• parameterized animation clues in the UI based on user-input, application- or system-state
• a reflecting, HDR-lit, shadow-casting map-view of your local area based on time of day and geographical position
• …

I could continue growing this list for some time. Just look at the UI of OS X Leopard (CoreImage, CoreVideo, CoreAnimation). Also Vista is well armed in that domain (Avalon aka Windows Presentation Foundation aka .NET 3.0 aka Windows Live Something). Now I can imagine some of you thinking: “Good lord… I get it. MacSlow wants to mimick the competition. What a faggot!” I’m sorry for you if you actually do think that. But that’s not the real deal behind it. We have most (if not all) of the needed low-level parts in place for this… cairo, OpenGL, gstreamer, librsvg, gegl (ok, not all are fully developed yet, but most are) to name some more well-known ones… akamaru, libclutter to name some lesser-known ones. What is needed is a general effects-, animation- and layout-library pulling together all those assests we have in the OpenSource field and become the icing on the cake in form of a convenience-layer. Also providing solid frameworks and standard boilerplate-calls for the usage of fragment/vertex-shaders, realtime image/video-filters etc.

That in turn should be used to conceive tools offering artists and designers to build UIs in a much more sophisticated fashion. Just like today design of an UI can be done in glade without a single line of code thus concentrating more on usability-aspects during the process. I would like to see people like e.g. jimmac not doing mockups of his ideas in blender, but for real and being able to test them out on real users… without him needing either learn all about coding gtk+, OpenGL, cairo and what not or waiting for some OpenSource-developer to find time and motivation to implement it. Or have the folks working on defining and testing the specs for Tango broaden their reach and also work on a common set of reasonable animation behaviours of a desktop UI. Things like those make all the sense (once we have the needed frameworks and tools in place). In my opinion this is something very worthwhile to undertake for Gnome & Co and the free desktop-systems in general. It does not stop on simple UI-matters. Having a standard set of shader-based effects for image-, video-, audio- and mesh-data provided by a common cross-platform library, makes the platform more attractive to developers, who are usually not very keen about Linux as a development-target at the moment. Aside from this, well-known OpenSource projects like Diva, PiTiVi, jashaka, jokosher et al. would probably also welcome such a tool set.

I would also like to see gtk+ 3.x, Qt 5.x, Gnome 3.x and KDE 5.x being able to base their future work on such a library. This would probably also be a welcomed effort by the Portland-project. Diversity is good… under the hood, but not on top, when you want to get things done. All Monos, Pythons, Rubys and whatever languages of the world will not give OpenSource-systems the boost in attracting new 3rd party developments, if we don’t boraden the disciplines our tool-chains can provide and simplify their utilization. This is especially of concern looking at the offerings on proprietary platforms. Big players like Sony, IBM, Sun, intel and should step in here to help with the heavy lifting ahead of us. The benefit for them is clearly a more media-complete solution with all the benefits OpenSource offers (I disregard the DRM-mess for the moment). Furthermore I would also like to suggest to the community to establish a good relation-ship with coders from the demo-scene. We need all the help we can get.

While this may not be needed today, it will be needed tomorrow and we better start working on it today to have it in a mature state in the future. No one can tell me that s/he would not like to see a project like this spawning. Of course it is a huge endevour… but to stop is to fall behind! It’s go big or go home!

Just my humble (but still very concerned and serious) 2/100 euro-cents. Comments welcome!

P.S.: These discussions here actually pushed the buttons in my brain to make me write up this.