It Came From the Past to Save the Future!
Forward: For a drawn out post on next-generation X rendering, this blog entry is really short on eye candy. I apologize, but I'm at home, separated from my beloved eye candy, and figured I should write this while I felt motivated. As a way of forcing my own hand, I'm making a link now to a blog entry I haven't yet written that will contain screenshots in the future :-)
Next-Generation Rendering For the Free Desktop
For the past half year or so Red Hat's desktop team has had people working toward making accelerated graphics rendering on the free desktop badass, but doing an ass job of actually talking about what they're doing in a larger public / GNOME context. They've been doing a combination of experimentation (from that cracktastic OpenGL compositing/window manager luminocity to xsnow for the Xcomposite generation) and knuckle-down no-holds-barred infrastructure work (like making Win32 GTK work on Cairo so GTK can move to cairo as the default backend). With RHEL4 kicked out the door we've been able to rebalance day-to-day work on GTK and X onto other people to give the nextgenren hackers free hands. Currently the full-time nextgenren team at Red Hat is Owen Taylor (gtk/pango maintainer), Søren Sandmann (x hacker), Diana Fong (visual designer), Kristian Høgsberg (x hacker) and Carl Worth (cairo maintainer).
I'm really excited because these guy's expertise is across a broad chunk of the rendering pipeline, from the toolkit down to the x server, which is going to give this effort the ability to work on this from a global perspective rather than optimizing the bits where we happen to have influence in. I'm doubly excited because other companies (well, Novell at least, but hopefully others will join) are starting to invest in this effort too!
I'm hoping to drag Owen into spinning this off into an umbrella effort (ala project utopia) to help maintain a coherent story/platform even as lots of people pour work into lots of different packages and distros. There are so many different ways to attack the X rendering issue that I'm a little worried about seeing a lot of fragmentation of effort and the result not being particularly coherent. I do hope people experiment with lots of different approaches, but I also really hope that in we can give developers a consistent platform for doing cool graphics on the free desktop. It would be a real shame to end up with the message in two years being "well, platform X has the feature you want, but you have to worry about also working with Y because X won't work well on distro Z". This sort of technology-choice morass can really dampen developers playing with this stuff and adding support all over GNOME, which is exactly the sort of quick-fiddling big-payoff stuff I think we'll see a lot of as soon as this stuff starts landing. In other words, lets push toward the point where people can feel confident and start hacking up cool things for this system inside GNOME.
What It Might Look Like
A really good system needs to have lots of pieces in place all hooked together....its not something that can be hacked apart and replaced by arbitrary random incompatible bits (though there are points of commonality, such as OpenGL or Render). For example the pieces in one imaginable architecture - by no means the decided-upon final one or anything - might look like:
* A sophisticated drawing layer (cairo using glitz/opengl or render as backends)
* Stock renderers built on top of that drawing layer (pdf/ps rendering backed by cairo - such as Alex Larsson's xpdf fork in evince, svg rendering backed by cairo, etc)
* A toolkit that agressively takes advantage of the features in the drawing layer, exposing them to applications and themes (gtk+)
* A window+compositing manager that can work closely with the toolkit but essentially takes the window contents as a static image in compositing (metacity with luminocity-like GL compositing manager features fused in to deal with window effects, synching up smooth resizing, live window thumbnailing, crazy pagers, etc)
* A hardware driver system to expose a low-level hardware accelerated rendering path to the drawing layer (opengl or render with hardware accel)
With that model we can implement things like:
* Toolkit themes that draw with layer blending effects, delightful bezier curves, and irritating alpha gradients
* Indiana Jones buttons that puff out smoothly animated clouds of smoke when you click on them
* Alpha transparency in applications whenever and wherever the urge strikes us
* Live window thumbnails
* Hardware accelerated PDF viewers
* Hundreds of spinning soft snowflakes floating over your screen.... without messing up nautilus
* A photograph of a field of long dry savanna grass as your desktop background... where the grass is gently swooshed around by a breeze created by moving your mouse across the background
* Windows that shrink scale and move all over the fucking place with cool animations
* Synchronized smooth resizing so there's no disjunct between window borders moving and the contents redrawing (you should see the demos of this in luminocity... it really makes a difference in how real the interface feels, just as double-buffering did for stuff moving)
* A shared path between on-screen display and printing (using Cairo's PDF/PS backends)
* Vector icons with very occasional super subtle animations rendered in realtime...a tiny fly which buzzes around the trash every several minutes, etc... think mood animations as in Riven (which as a total random aside is still a shockingly beautiful and atmospheric game years after it came out, postage stamp sized multimedia videos notwithstanding)
* Workspace switching effects so lavish they make Keynote jealous
* Brush stroke / Sumi-e, tiger striped, and other dynamically rendered themes where every button, every line looks a little different (need to post shots / explanation of this stuff, but another day)
* Progress bars made with tendrils of curves that smoothly twist and squirm like a bucket of snakes as the bar grows
* Text transformed and twisted beyond recognition in a manner both unseemly and cruel
* A 10% opaque giant floating head of tigert overlayed above all the windows and the desktop.
* etc etc. In short: awesome.
And that's a conservative approach to this: each window essentially renders into a texture which are then combined in a separate rendering pass by the compositing manager. A lot of the work Diana does challenges our assumptions about what this rendering system should be able to do. For example, something as simple as a swoosh that cuts across both the window and the titlebar is currently very tricky. Diana's work has illustrated something that may be obvious, but seems to be forgotten in the excitement to build the One True Graphics Pipeline (this does not exist!): Its very important to figure out many of the things you want to do with the graphics system before you get in too deep and dirty, because there are a lot of directions we could go that call for rather different architectural choices. To give one example, if we decided we really cared about having lots of animations throughout GNOME (this isn't something we're pushing, but we talked about it) that would dictate a very different approach from a graphics system where we really really cared about printing. You can't always have your cake and eat it too... especially not when you consider implementation constraints.
Another example of how prioritizing "what do we want to improve with this" can change the direction: Since taking advantage of these new toys would require a new theme system, Havoc and I have been talking about how a very different theme / widget rendering system might work with this that allows for custom design of any window, widget, or anything in between. One of the things us designers have been experimenting with behind closed doors is what you can do with a window's design when its not drawn out of a bunch of stock widgets but you have a freer hand. (This does not mean visual inconsistency, just as a magazine can maintain a consistent look but still do a fresh layout for each page using a mix of stock and new elements.) The results can be really good. No matter how good the artist, you can only get so far designing a crude palette of some fixed number of widgets which are then used in preset. A good theme/widget rendering framework would help us negotiate this balance between re-using stock elements, and overriding the rendering of widgets at appropriate points to customize how a "Control Center Preference Page" is drawn or to simply shift the text in buttons over 10 pixels to the left. Figuring out how this stuff works, or if we just want to leave the theming issue alone (which would sort of be a shame given how much of the old flooring we're tearing up around it), may also have a significant impact on the final architecture.
A radical model (which also avoids multi-pass rendering without opening up security issues present in sharing direct access to existing graphic cards between processes) might involve a centrally rendered scene-graph where each client is given a subtree to add higher-level primitives. That could give us access to candy like pixel and vertex shaders (which we experimented with several months ago as part of rendering subtle but live backgrounds of grass fields, etc), which are attached to nodes on the render tree. Of course, there are many paths for leveraging shaders short of a full scene graph system. The scene graph model has a lot of significant concerns that are not as relevant to, say, 3D games where this model is common. Text rendering is one example.
Owen and company have slides from the X dev conf, but the punks did them as SVGs so unless you have their k-rad Cairo backed SVG slide presentation program, or if you're willing to view slides in Inkscape... they're not much good (though it is cool that you can find the slide you need using Nautilus thumbnails, but I digress) (hmmm, you can also open them in eog). Honestly, not the most inspiring OR detailed slides in the world either. I don't think they'd had much sleep when they wrote them up. *grin*
Anyway... I'm rambling. I've given a couple points too much depth, most points not enough depth, many points I've missed, and doubtless some I've gotten wrong, but I knew if I waited to write the perfect post on this there'd be only more backlog of material to share... so a braindump it was. :-) I guess in the end I'm pretty excited. It feels like we're running the last couple miles to get to the giant great-rendering payoff Keith Packard kicked off in the X world several years ago.
Code and stuff
* Cairo I think everyone knows about... writing for Cairo in Python or Mono is especially cool. Its really easy to get something that looks good going in short order. If you haven't played with it, you should!
* Luminocity is in GNOME cvs with the module name 'luminocity'
* Metacity compositing work is in 'metacity' with the branch 'spiffifity'
* GTK+ / Cairo integration.... gtk+ HEAD!
Apparently they also have a jhbuild setup that'll build all this stuff thats headed for CVS in fairly short order.
And for my last point...
作者：jcome — 上次修改时间: 2006-04-04 23:15
就如 Seth Nickell 在他上一篇 blog 上承诺的，他已经放出许多体验快照和视频，以及一步一步的试验步骤。请点击这里 ，还有 这里 是我们论坛的讨论。
着RHEL4的推出，他们可以重新分配 GTK 和 X 的日常工作，让这些忙于下一代渲染技术的黑客腾出手来。目前红帽子全职从事这个下一代 渲染技术的团队成员有Owen Taylor(gtk/pango 维护者)，Søren Sandmann (X黑客), Diana Fong (视觉设计师), Kristian Høgsberg (X黑客) 和 Carl Worth (cairo 维护者)。专家意见是要在广阔的领域中穿梭，从工具集一直到 x 服务器，用一个全局的视野来做这个事。就像 utopia 项目那样，（jcome:不要说你不知道utopia哦。）
* 一个完美成熟的绘图层（使用glitz/opengl 或 Render 作为后端的 Cairo）
* 在绘图层之上构建的 Stock 渲染器 ( cairo 的 pdf/ps 渲染后端 - 比如 evince中的 Alex Larsson的 xpdf fork , cairo的 svg 渲染后端,等等)
* 一个窗口＋合成（compositing）管理器，(带类似luminocity 的 GL 合成管理器 的起到处理窗口效果，同步平滑的最小化，实况的窗口缩小图（thumbnailing）
, crazy pagers, 等等)
* 一个硬件驱动系统，用来向绘图层揭露底层硬件加速渲染的途径 （opengl或带硬件加速的 Render）
* 使用多 层混合 效果的工具包主题
* Indiana Jones 按钮，当你按下是会冒出淡淡的动态烟雾。（jcome:这个印地安妮琼斯是游戏吧？我
* 同步的平滑的大小改变在窗口border的移动和窗口内容重绘之间不会脱节 (请参照luminocity的演示。。。只用了双缓冲来处理移动，就产生了完全不同
的界面感受） v屏幕和打印共享的路径(用 Cairo的 PDF/PS 后端) （jcome:是说显示是啥样，打印出来就是啥样。这有点言过其实，我对此还有点知识）
嗡响。等等。可以考虑 Riven中的情态动画。 （看官：Riven是什么？ jcome：I dont know :) ）
* Brush stroke / Sumi-e, tiger striped, 以及其他动态渲染的主题，每一个按钮，每一条线看起来都有细微的差异(need to post shots / explanation of this stuff, but another day)
* 螺旋曲线一样的进度条平滑的缠绕蠕动，像蛇（jcome: 啊？）
* a 10% opaque giant floating head of tigert overlayed above all the windows and the desktop. - 等等.
目前来看，compiz 进入 fedora，cairo 的新版本，clearlooks 的成果都还不错。但是，为什么总给人一种“革命尚未开始，大家耐心等待”的感觉呢？
奇怪的是，这一个 Seth 发表了大作之后，BLOG 就搬到了 http://blogs.gnome.org，然后又过了半年，就不知道又搬到哪儿了。他该出来总结下的。找到这篇文章，搜索关键字是 Diana Fong :D
BTW, http://www.hula-project.org/Hula_Project 是什么好玩具？