LibreOffice Conference 2026: User Interface Evolution for Apple Devices

[music] Hi everyone, welcome. My name is Dan Williams and I'm a a document foundation engineer uh recently hired to work on user interface and Mac development. Today I will take you on a quick tour through the history of the Mac port uh and the evolution of Libra Office on the Mac and then I'll cover the current state of the Mac port and what we can do going forward to give Apple users a better experience on uh Apple platforms with lever office. So let's begin. So this is uh long a timeline of long long ago in the very beginning. Um you'll see here basically in prehistory openoff.org was first open sourced in 2000. Uh and then in between there and the actual release of Mac OS 10 there was a lot of interest uh in the new Mac OS at that point before it was even released. Uh so we had Patrick Lubie and Babach from Sun were working on a native uh Mac OS 10 port at that time. uh they were doing things like uh event loop integration, doing some basic graphics work. Uh that kind of stuff. It was uh Mac OS 10 itself was pretty rough at that point. Uh the public beta was somewhere in here. Uh and that was very rough. Uh and so I don't envy the work that Patrick and Babach and others uh at Sun at that point had to do to get things working to any particular point. This work was never completed. So uh you know quite difficult. There wasn't a lot that they could realistically do at that point. So Mac OS 10 1.0 is released uh in mid 2001. And then finally um you know about a year later or so we have Open Office 1.0 the final release. Uh and then a little bit after that we have the first beginnings of an actual Mac port of openoff.org or that works and that was based on the uh x11 version uh for Linux but essentially and Unix I guess because at that point somebody correct me if I'm wrong but uh open office also ran on Solaris um and the Macport used the X11 code and that was basically a way to make things work more quickly on the Mac because we could use all the existing code uh and we did not have to do a whole bunch of custom code for Mac OS 10 itself. So, what did that look like? Well, this is a a screenshot that I found um of what it it did look like. It used the normal uh Star Office and Open Office controls. Uh those were obviously did not look anything like the Aqua user interface that the Mac had, but you know, at least the Windows were native. There was some other uh small integrations there that got put together. Uh but everything was basically exactly as you'd see it uh on Linux or even on Windows. It was never going to be easy to use. It was never going to integrate well with the rest of Mac OS and it was basically a dead end, but it was a way to get there very quickly. So then next what happened uh in the UI side of things was around 2004 the native widget framework came out. Uh I had a hand in developing some of that when I was working at Red Hat early on as an open office developer. And what the native widget framework was supposed to do was also kind of be an easy way to get towards native looking controls even if it didn't feel uh particularly like the native OS. But it got a good good way there. Basically, we recognize that all platforms have some way of drawing what a control looks like. And so if you add a layer to kind of abstract that out and define there's, you know, a button control, there's a an a combo box control, there's an edit box control that has a couple of different parts in there. you can tell each platform to draw those pieces uh of the user interface and then copy those into uh the open office windows. And you can see here this I think is a a KDE version perhaps. Um but you know you can see that the controls here look much different than the previous ones that were the the default uh Open Office theme. So that debuted with Open Office 2.0 0 and I think the first users of that were GTK and KDE. There was a lot of interest around that. I had some early screenshots long ago on my people.redrered.com site, but when I left Red Hat, that got deleted and I forgot to back it up. So, that is lost to history. Next, we finally get a Mac OS 10 native port. um somewhere around 2007, work restarted on the Aqua VCL plugin, and that's the the native plugin that we still use today essentially for the Mac OS 10 port. Uh the first native release was Open Office 3.0 and that used native widget framework. It used a mix of Coco and Carbon. It used the Apple type services for Unicode, that's the ATSU, uh, for font detection and font metrics and those kinds of things. And, you know, it doesn't look so bad. It obviously doesn't look like a native application, but it gets a lot closer than what we had before. Um, and that was a lot of hard work uh from a lot of people. Um, but I think it it helped increase the perception of Open Office with Mac users. So then we jump forward a number of years and Quaon starts doing dialogue welding. The problem that we had uh or the problem the project had at this point was and at this point it's Libra office no longer open office was that a lot of the controls in the user interface was essentially handcoded or you know pixel positioned in the code. there was no good way to lay out the user interface more easily. Use graphical tools to do that. So, the welding project uh decided to use graphical tools like Glade, which is a fairly well-known GTK graphical interface designer, uh, and be able to lay those files out, write those details to an XML file, and then Libra Office loads in the XML file and is able to create those widgets that the XML file specifies. It's a lot easier for people to change the look uh of Libra Office and add new user interface elements because you can do it graphically instead of all over the code. It's closer to native widgets than before. And inside the code, there are actually native widgets created and placed in a widget hierarchy for the different uh toolkits. But when it comes to the Mac, uh the Mac doesn't actually use any of that. The way that it works on the Mac right now is that it still goes through the normal uh Libra Office uh control setup and then draws native widgets in place of uh where the rest of the uh cute VCL plugin and GTK etc actually draw their own native controls. So it could be useful. It is useful in some cases but the Mac doesn't really take advantage of it. So where are we now? We've kind of reached a point with diminishing returns uh on the native widget framework. We've gotten as far as we can kind of go with that. Uh and for the past 10 years or so, the focus has been on keeping up with changes that Apple makes in the operating systems every year, fixing bugs and uh better integration with standard Mac features. But the user interface on the Mac side itself hasn't really had any significant advances since uh almost 2008 um when the native widget framework finally debuted for the Mac port. So we'll dive a little deeper into that. What the native widget framework for Mac does um is that it literally takes there's there's one control, one operating system or API object. There's one for a button, there's one for a combo box, and those are kind of offscreen somewhere. Um long ago they were literally offscreen. Uh but these days are just in memory. And every time that Libra Office wants to draw any of the controls, the code will take that one, resize it, set all the attributes on it. Is it disabled? Is it selected? Does it have a focus rectangle? Uh what's the background color? It'll change that one control, take a picture of that, and copy it on the screen. And so you know essentially there's a kind of box over here somewhere that you don't see and we just draw it copy it and then the next one change all the the uh properties draw it copy it and that's how the entire interface is drawn. It works you know but it's not not great. There's also a lot of old APIs. There's still a few bits of carbon. Carbon is the old framework that took a lot of the classic Mac OS APIs from the 90s and moved those into OS 10 and updated a lot of the compatibility issues that it had. Uh so it's kind of a subset of the old classic Mac OS APIs ported to OS 10. There are still two pieces of Carbon in Libra Office's Mac plugin right now. Um, and those pieces are uh there's some stuff for pop-up menus and then frames. Um, probably won't be super hard to remove those. Uh, but you also don't need to because Carbon exists and it works. Uh, but it's almost 30 years old. Carbon and Apple of all companies, you would expect to kill backwards compatibility. They haven't done it yet. I expect they'll probably start getting rid of carbon fairly soon. So, we probably want to change that. Uh the also the Apple remote code depends on these old frameworks as well. And that's for things like uh capturing global uh keystrokes or mouse um buttons, mouse events. Uh and that's one thing that we need to either drop completely because I don't know how many people have those old Apple remotes anymore. Um, but that's the third place that still uses some of the old APIs. There's also a pretty creaky event loop right now. One of the problems is that um bridging between Mac OS's events and the internal Libra Office uh event loop can be uh a little problematic. There are sometimes re-entry issues. We recently had a bug where just dragging the window around then Mac OS would try to snap the window to full size that triggered a deadlock inside Libra Office that was somewhat difficult to solve. Uh that particular area of the code is not great. So it is very fragile due to some of those uh those state mappings that we have to do between the Mac OS side uh and the internal Libra Office side. Then we also have to deal with every year Apple comes out with a new operating system version and every year they change some things. These are some of the older uh versions from you know like late 20s uh 2000s and mid2010s. It's kind of a progression of uh what kinds of changes there are. You know this the change from here to here. Maybe the metrics didn't change that much, but uh you know, I mean, from here to here to here, these bars are a lot different. And so sometimes we need to tweak things on a per pixel basis to account for that. Another issue was liquid glass recently. Uh I know Patrick had to do some like this gets two more pixels when we draw it and this gets a few less pixels when we draw it. Um it's not that's one of the things that has to happen every single year that takes a little energy away. There's also dark mode. Um you know I know that Libra Office has issues with uh operating system themes and dark mode light mode on a number of different platforms. Mac OS is no different. That was introduced with Mojave and that causes uh problems uh throughout the codebase. So, what kinds of issues do we see just in general today? Uh, first, the look and feel is not exactly where we want it to be. You know, it's not bad, but we've been living with it for a long time, and it um could work better. Uh, also, every year when Apple does the operating system refreshes, they come out with new features. Libra Office tends to lag those features fairly significantly. Uh, I'll talk about some of those a little bit later, ones that I think might be useful for Libra Office users and uh, ones that probably wouldn't be too bad to to implement. U, but, you know, there's only a few people who are working on that area of the code. Uh, and so any help would be appreciated on those. Uh, there's also, like I talked about the event loop, there's bugs in odd places due to some of the translation layers. Uh, and one other thing that has come up a lot recently is some of the key bindings aren't quite consistent with what Mac OS does. So, Libra Office has default key bindings for moving the cursor around without using the mouse for, you know, in one case. Uh, but those aren't quite the same as what every other uh application on Mac OS uses. So, with all that, what comes next? Well, let's find out. One option uh that I've been exploring recently is to use the well-known cute uh platform or excuse me cute crossplatform framework. Uh there is already an existing plugin for Linux and that also because cute is crossplatform it works on Windows it works on Linux it works on Mac it works on mobile devices as well iOS and probably Android I don't know for sure Android. Yes. Uh what happens if we try to clean that up and use that for the Mac? Well, one of the things that would be a pretty big benefit there is that a lot of those issues and some of those gaps, especially around event handling and some of the graphical stuff, they kind of go away because the framework handles that for us and we don't have to write specific code in Libra Office to deal with that. It also gets rid of a lot of code uh in other places that we can just let cute handle. Um and if the visual results of that is similar to what we already have, it seems like maybe that would be a win from the uh developer time perspective. If we can spend time focusing on things that uh are more important for Mac users or give you Mac users a better experience as opposed to some of that creaky old stuff down in the guts, but we can also do Mac specific things still alongside Cube. What are some of the issues with that? Well, we'd need to do some upstream work uh in the Cube project itself. One thing that I was looking at recently was pop-up menus. Those are still not native pop-up menus the same way we currently have with the Mac port. Um, so we'd probably need to enhance cute itself. Uh, and it's still not using real native widgets. The cute plugin does use welding that we talked about earlier, so we might get a little bit further there. Um but also uh they're still not quite what we could do ourselves internally. And also uh using cute does not solve a lot of those feature lag problems that I talked about earlier because they don't always update cute for these things immediately when Apple does either. But that's not a net loss for us because we haven't been doing that in the Libra Office project ourselves. This is one uh this is kind of the current state of what the cute VCL plugin looks like. On this side is the cute one. On this side is the current uh Libra Office Mac native aqua version. You know, you can tell there's some places that we need to fix some things up and the spacing isn't quite what uh what we want. I'm working on that. Others are working on that, too. Uh but it doesn't look that different. So perhaps if we can get to the point where they look similar enough, great. Maybe that's a path forward. Otherwise, we could do dialog welding with Aqua. It gets us closer to a fully native look, but it is also a huge chunk of work because we need to develop all of that. We need to um essentially duplicate a lot of the logic that the cute plugin already does uh to do the welding. and it still uses native widget framework for some parts anyway. So, it's a net addition of a huge chunk of code to the Mac side and the Aqua side um and a maintenance burden. So, I I'm still on the fence. I'd like to try it. I don't know if I'll have time to try it, but honestly, I think it's probably not the right direction. Another thing we can look at is mobile platforms. Calabra released a downstream app for iPhones a number of years ago. U but we're also talking as uh the upstream Libra Office project. What would it look like for first party or upstream mobile apps uh on uh app stores for we already have an Android one I believe but that's just a viewer. Is that correct? >> Impress viewer. >> Impress viewer. So, what would it look like if for Android and iOS, iPad, whatever, iPhone Duo if you happen to have one? Uh, what would it look like to have a a much more functional application on mobile platforms? That's also a huge chunk of work. And it's obviously going to need a completely different user interface because first off, you're touching it. Uh, you're not using a mouse. And so, our UI elements need to be bigger. They need to be more simplified. Uh but I think it would be pretty awesome to have a lot more of the functionality on iPhone or iPad uh as we already have uh on the desktop. But kind of a little more down to earth, what kinds of things could we do on the Mac side that people can actually use in a relatively recent or relatively short time frame? Well, there's a number. First off, let's talk about AI. Libra Office, I guess, takes a somewhat cautious approach to AI. It does not try to force AI on you, and it is all up to choice. You can decide to use uh AI through extensions and plugins if you want. Um, but you know, there's no little AI assistant down in the corner trying to make you use AI all the time. How does that apply to Apple platforms? Well, something Apple released a number of years ago are AI writing or Apple writing tools and that is a mostly ondevice framework that is throughout the entire OS and you can select port or parts of text and have the writing tools change that text for you, proofread that text. Um, I've seen a number of users asking for this. Uh, and it turns out that it's probably not too hard to do, but it does require a number of hooks into the Libra Office text engine that do not yet exist for any platform. Uh so maybe we could do those and make that kind of thing available uh to other platforms because what Apple writing tools do is they try to reach into your document and when you tell it to change something they process that and then they want to update the text in your document. So that's not too bad. And these are some examples of this is the the talk description uh that I have and that's what it looks like when it's processed through some of the Apple writing tools. You can decide whether the Apple writing tools are are better than I am or not. Uh there's also image playgrounds and that's uh you can type and change uh tell the system what kind of image you want to create. uh you can have it go out to chat GPT. This is also kind of a fairly encapsulated code and a bolt-on. So you can see here you can just kind of pick some themes and describe something and then you get a picture out of it. You know, it's OSwide. Why not support it if it's not that much? There's also continuity sketch. This one I think could be pretty interesting and it also is a fairly easy bolt-on. um you draw on your phone or an iPad with your finger or a stylus and then you can add that sketch into your Office document. Um it's pretty easy to the the flow is pretty easy on the Mac and I think that would be a pretty good uh thing to support. There's some miscellaneous things. There's Siri intents which is a new thing from I think a year or two ago. Um, and that's where you can have uh you can declare operations that your application can do and then when the user asks Siri to do something on the Mac, uh Siri can figure out how to control the application uh and reach in and do things to your document or do things to your text. There's also we could use system spell and grammar checking. That's a choice Calabra made with their uh iPhone app to use the system spell checker instead of the integrated Libra Office one. Maybe that's an option. Um and then also there another interesting one would be the spotlight importer. Uh if you have Libra Office documents, I don't think these days you can search them with spotlight. Um also if you load them onto your phone, you can't really see a preview of those documents or what's inside. Um you know, so these are some kind of smaller things that I think would be uh pretty useful for Mac users. But I'm only one person despite what you've heard. I'm not a coding god. So, it takes a community. And if anybody wants to help out on any of these things, I'm very willing to mentor or do code reviews, that kind of stuff. And I would be interested in working on some of these, but it would be great to have some help as well. And if you have any other ideas, please send them along. Thank you. I think we've got maybe a minute or two for questions. I have a question. >> It's interesting you mentioned the Apple supplied AI tools which um if we make them available on to users, we may have to do it in a way that they are aware that they are on device or I don't want to be in a position where we're simultaneously marketing Libra Office as having no built-in AI. Then they install it on Mac OS. Then they right click on the text and it says rewrite it and they'll say what the hell where did that come from? So I reticent to put more and more and more options in but maybe some kind of first start. Do you want to use Apple's built-in >> AI and that's something that we have to specifically enable and so we could gate that you know in front of a some kind of notification. >> Yeah. Yeah. For sure. One last question. >> What drives what sort of feedback hang on what sort of bugs or feedback do you get from the Mac OS user using Libra Office or what prevent them from is it related to the u Mac OS UI as you described or is it related to I would say conversion from office to lial office? I would say most of the feedback that I see and maybe Haiko can chip or chime in there too. Most of the feedback I see is not really about the user interface. It looks okay for most of the users, but it's often about how it works. You know, in a very uh micro basis, you know, the key, like I said, the key bindings aren't quite what they should be. Um, you know, [snorts] other Mac specific bugs. Um, but that might just be because I'm only looking at the bug reports and it's not what general community discussion is. Haiko, do you have any quick thoughts on that? >> Well, I think the UI works pretty nicely well and I don't recall any specific issue about UI. There might be a few, but it's nothing that is of major concern to user. But general we have a lot to do and we introduce issues