Flutter answers questions about more robust UI.
It's good that you chose the right tool for the job and more people should know that there are options. But fundamentally I'm most motivated by the possibility of a robust UI framework made from first principles to be as low friction as egui but with the accessibility, performance, and visual flexibility of stylable retained mode guis.
Raph Levien and the xilem project might be getting us closer.
[Edit: although the standard accessibility criticisms apply to my application; although that's more of an issue with my implementation than an indictment of immediate mode generally.]
Imagine you have a to-do list of 100 items. What happens when a remote user drags and drops item 100 to lie between items 1 and 2? In retained mode, that's clearly communicated to the UI toolkit; the widget representing that todo is told to change its position in the list. In immediate mode, you can't just destroy and re-create the a11y tree on each render. You need some kind of tree diffing algorithm to figure out that it's one op (move item) instead of ~200 ops (change the name and checkbox state for all items between 2 and 100).
You don't need to do it each render, just when something structural changes, right?
You don't lose anything.
This is but my opinion, but toolkits tend to be opinionated piles of **. Talking directly with the GPU to paint stuff is often just saner to me.
Over my years making UIs I've found most of the bugs you get is due to incorrect state handling in applications. Having a framework you use which is opinionated and helps you solve that is pretty nice. (If your UI requirements are such that you need it.)
This can be problematic, e.g. some of the sensor interfaces I have, I want to always display correct data, even if not focused. So, I have to decide if I want to have old data shown in the background misleading users, or have a per penalty from constant renders. Or try something else to be clever. (Maybe have it update at a low rate if not focused? I think that's the move...)
sounds analogous to manual memory management
This is also mentioned in the gui docs here https://github.com/emilk/egui#why-immediate-mode:
> egui only repaints when there is interaction (e.g. mouse movement) or an animation, so if your app is idle, no CPU is wasted.
Reactive mode is the one you are looking for.
I recall the author posting an imgui update saying this will be an officially supported mode, but AFAIK it's still not the case. Otherwise I would be building all my applications with imgui going forward.
Re-rendering the screen, even if it's fast, incurs a lot of memory bandwidth to draw everything and swap framebuffers etc. Not something you'd like to happen on mobile, in particular. Just because the waste is "small", doesn't mean it's acceptable.
By the way once upon a time, visual studio code I think it was, was using like 20% cpu when idle just because of the blinking cursor, fun.
It's not perfect, but I don't know if there's much on the market that addresses robust UI and single code base as well as Flutter.
Very open to other things that are not more complex than Flutter to accomplish the single codebase to multi platform solution it does provide.
> egui is great for projects where the application runtime is short lived
Accessibility and keyboard shortcuts are of extreme importance.
I'm a bit concerned with Egui gaining in popularity for general purpose GUI applications. It's leading to feature bloat, and probably more overhead. The stated goal originally was that Egui should use less than 1% of main thread frame time.
Originally, Egui was completely one pass. The API looks more general than that; you can align things against the bottom or right, and get things above or to the left to adjust. But originally, that didn't work. You pretty much had to lay out widgets down and to the right. This is fast and simple. Lots of stuff didn't work, such as scrolling text boxes with line wrap.
But users doing ordinary GUI work are demanding more and more layout features, and won't stop until they get browser level layout. Overhead is increasing.[1] This is a problem in Rust game land, which is tiny. At some point, someone may need to fork Egui and create Egui-lite.
If you want a simple, one pass layout that super fast, you use the fast-and-dumb-on-purpose-engine. If you want a fancy dynamic scaling layout with breakpoints and blah blah blah, you use the whole-damn-browser-engine.
Thinking along the lines of geometry managers in Tk. Or wxWidgets Sizers.
https://wiki.tcl-lang.org/page/Geometry+Managers
Maybe I am missing something basic and this won't work/doesn't address the problem.
But maybe there's good prior art to learn from here.
If your UI lib is not doing scrolling text boxes with line wrap, can't have different alignments, doesn't have a solution for context menus, then what good is it?
I remember that I essentially had to code it myself a few years ago, but I can't remember if it was egui or imgui.
I guess you worry that the feature creep dilutes the maintainers' focus, but opening it up to a broader audience might get access to more contributors.
To some extent, yes. Here's an example.[1] So many features have been added that the unit tests were taking too long. So tests were switched to use a faster allocator. That won't even compile for cross-compilation from Linux to Windows.
Because tests won't run, chasing down other cross-compilation bugs got much harder.[2]
This is the price of feature bloat. Core stuff is breaking and not getting fixed as cool features are bolted on. Currently, 799 open bugs. The technical debt is building up.
1. People love it because it's lightweight and fast.
2. More and more people use it.
3. Feature requests start to roll in.
4. It becomes bloated and slow.
5. People get fed up and start to hate it.
6. Create a new ticketing system and return to step 1.Back in 2023, this created a bug where a text box was misaligned on alternate frames.[1] Amusingly, I tried to capture a video of this, and the 30 FPS video looked perfect, because it was capturing only alternate frames of 60 FPS refresh.
Bugs in layout can be very strange in Egui.
Did you just search for bugs containing "CPU usage" and not actually read it?
Originally HTML was one-pass. Until they added tables. And tables require at least two passes to lay out.
There's only so far that you can take one pass rendering
Right. The whole point of Egui was supposed to be that it was a game renderer, a text and button overlay on the game graphics. Not the entire screen. That's why it's immediate mode. It's supposed to redraw on every game frame. Most UI programs don't need or want that.
But it was better at the basics than anything else in Rust, so it caught on for routine non-game programs. And here we are.
Seriously. The main tale I think is how awful Rust's story on GUIs are such that most Rust users prefer a pretty awful GUI library because it's somehow better than everything else. And that's coming from someone who's favorite language is easily Rust.
In other words, with default styling you have to calculate natural widths of every cell in the whole table before you can begjn layout out even the first cell of the table.
There are style settings you can use which change this, and allow layout of early cells or rows to complete without depending on later cells.
(Also the height of each row can depend on the height of the content of every cell in the row. But that's local to each row so it's a relatively small dependency.)
I guess it makes sense since immediate mode focuses on speed and applications like games, but if only there was best of both worlds.
Because web's "native UI stack" is almost non-existent for actual UIs. This has started to turn for the better only very recently: https://open-ui.org/
Flutter, which does its own rendering of controls, needs to implement a lot of accessibility features by itself.
For my part I have some accessibility needs that typically aren't well supported by immediate mode GUIs. And my personal take is, this is 100% fine. If you're working on a thing where an immediate mode GUI toolkit is even a realistic option, you're probably also in a situation where it's exceedingly unlikely that making your hobby project or prototype or whatever harder to work on will cause a net increase in humanity's overall sense of wellbeing.
And if something comes up and you do end up needing to make changes for someone else's benefit, hey, good news, immediate mode GUIs are really, really easy to hack on.
Not sure that I'd recommend this approach to everyone. Protobuf code generation can be finicky to set up, but I'm doing it so that I can access go's rich array of libraries in my app.
My music model is all Protobuf messages, which go from Dart/Flutter land to Kotlin/C/Swift/JS audio backends on target platforms. I also use Protobuf for saving and sharing. It’s been incredibly resilient and performant.
I've been playing with the idea of creating a "protobuf db" library that would allow you define schemas in protobuf and then query them with something akin to an ORM. It wouldn't make any sense for large databases, but for embedded applications that only need to store a few MB of data, it would be perfect.
I've been working on a separate branch, which finally builds (there were 1100+ errors), but I'm still working through iOS/macOS build things for it before merging it to main. (I've sadly had to abandon the Android build, because Google Play was a comparative pain, and FluidSynth upstream kept breaking the Android build I set up for them. But I'm reviving the project for iOS, macOS, and web at least.)
Here's the branch: https://github.com/JonLatane/BeatFlutter/tree/update-to-late...
I used proto buf with rust, I had a rust client that spoke to my flutter frontend via dbus. The rust client connected to my remote server via a web socket and all messages were wrapped in protobuf and sent as binary. Made everything a lot more concrete... But it basically forced me to build my own much shittier version of gRPC. Since, if the wan for your network was every killed the client was notified too late and you'd end up with missing messages if the network buffer got filled. We added a message id and acknowledgement process with sqlite backing up each message.
I still have nightmares about why I built that.
I used to use protobuf but now I just use JSON, over stdin/stdout on desktop. It’s honestly quite good.
Personally I wish we could just use UNIX sockets for "localhost-only TCP", but software support is just not there.
I do wish gRPC allowed for easy usage of UNIX domain sockets and perhaps named pipes, however. Sometimes all you need is IPC, but in my case, I'm happy to have remote usage builtin.
That said, a random password should be enough protection, even if it isn't the cleanest solution.
[0]: e.g. https://palant.info/2020/06/22/exploiting-bitdefender-antivi...
Can you expand on this OP? I've never had problems with `setState` nor "lasagna code" in Flutter. From a quick search I mostly seem to find questions from people who are still learning Flutter and getting basic things wrong.
With egui you pretty much don't have to think about it at all.
Pros
- Solid widget set
- Easy to get started
- Less state management
- Easy to make custom widgets
- Active community and crates (e.g docking view, tables, etc.)
- Fast to build new Ui
Cons
- Harder to do layouts (has multipass and some flexbox crates but still hard and compile loop makes it slow to iterate)
- Bring your own architecture (no restrictions on how you build your app so easy to make spaghetti if you’re not careful)
Egui is currently my favorite Rust UI crate although Slint and iced are also interesting.
No quick and easy drag&drop just yet, but IDE support for live preview rendering makes it come pretty close. I do long for the Visual Studio GUI design days, but things aren't as barebones anymore as they used to be in open source Rust land.
When hand-writing XAML or similar, it's great to see the UI created live. Like editing markdown and seeing the preview, versus editing markdown and not seeing the preview.
We aren't talking Dreamweaver here.
IMGUIs are almost always relegated to debug UIs and single devs because its actually terrible to maintain something that binds presentation and business logic like IMGUIs often do (and when they don't they lose the simplicity).
You can see the demand in the sheer number of WYSWYG editors for the web.
But for development, basically all the big players stopped trying or died for other reasons. I just think no-one's got the will to try it.
I think it could be a huge opportunity for someone. Right now, with AI coming to the fore in development, seems to be when it would become absolutely killer for less code orientated people making their own apps by adding/dragging controls around and telling an AI what each control should do. All without a programmer involved. The AI could even "solve" the hard problem of a good responsive WYSWYG editor by making assumptions of how the user probably wants the controls anchored.
So I think that's the market we'll see a WYSWIG editor emerge again for.
I would add that a large part of the push away from it came from not having dedicated design and development teams. Is, I think, why industries such as gaming have stuck to a lot of these workflows. You have art designers creating the assets and hand them off to an integration team that will get them into the game. Tooling is specifically made to integrate the art and the program.
In the web, we seem to have tried to converge all of that tooling into the symbolic text. Works great when it can work. But it greatly limits what you can graphically do. And is largely why we don't design things graphically anymore.
These all go direct to code instead of XML or some other extra layer of code.
What I would probably focus on is better integration with Figma and similar tools. Use that to do the WYSIWYG part and then generate corresponding code (possibly with LLMs).
The biggest limitation you tend to have in WYSWIG is that at some point you really need to have the true data the user sees to ensure everything looks good. That becomes a bigger hassle than coding the UI manually. (Particularly when doing multi-platform things.)
Like, why aren't more big projects use accessibility as hot reloading if those are so great?
Or, a simple illustration to your reliability point - part of the reason the files are hard to read is because XML is an atrocious format. And also tools like VS aren't even smart to preserve user formatting, so you can't even manually make it easier to read. Why hasn't it been changed in so many years?
How is shipping egui apps vs flutter. I'd imagine that especially shipping a rust integration with Flutter might be a bit of a pain
iced has some nice looking apps written using it, but they all seem "laggy" on my high-end Linux box somehow. I'm wondering if this is a limitation of the immediate mode GUIs, or something related to my system, or an issue in the apps?
For example, drag-and-drops or selecting text with mouse lag behind the mouse cursor.
I'm wondering if I'm not holding it right somehow?
which apps are laggy? iced is fast AF. try running the game of life example on release and crank to max speed. there is literally no lag
could be something wrong with your GPU. join the Discord and we'll help you debug.
EDIT: I'm told you could be having vsync issues. try `ICED_PRESENT_MODE=immediate`
https://github.com/slint-ui/slint/blob/master/FAQ.md#licensi...
I only mention this because those constraints are notably more restrictive than the vast majority of the Rust crate ecosystem.
I take no particular stance on whether this is a fair or good practice or about the technical suitability of Slint beyond this concern, I just think it's a hurdle for most people so they should be made aware early and often.
Honestly, it’s a bit sad that for some, using libraries for free isn't enough, they also expect to do so without even mentioning the project.
For my use case it was easier to get up and running with a TUI toolkit (ratatui in my case). Plenty of limitations there too alas and I finally landed on Qt…
However, updating the DOM and then turning the DOM to an image (i.e., rendering it) still has an indirection that using canvas/webgl/etc. don't have.
Both Mithril and Preact use virtual DOMs:
I had the impression that the lack of double buffering would imply a more direct access, this the "immediate" in the name.
[1]: https://github.com/ocornut/imgui/wiki/About-the-IMGUI-paradi...
Makes sense to use a primitive instrument to satisfy primitive needs, those eschewing a lot of extra complexity.
Though if some funding does fall through for better gui, isn't that a risk of having to do another switch?
mobile and desktop are not different platform but different devices altogether that warrant separate designs IMHO
How well does it integrate with Androids APIs or even IOS, can I make a full blown app with it. I believe the main reason why people use FFI to interface Rust and Dart/Flutter for mobile development is because Rust does not have an Android/IOS framework with the higher level APIs it just has native APIs (fs, io..)
I liked the DX with the tools and the `rsx!` macro. The use of `#[cfg(feature = "server")]` to define server-side code is interesting, it lets you keep a shared codebase for frontend and backend, while still controlling what gets compiled to WASM for the client.
[0] -- https://dioxuslabs.com/
[1] -- https://blazingboard.ch/ (not mobile friendly, sorry)
Non-goals
* Become the most powerful GUI library
* Native looking interface
valgrind --leak-check=full --show-reachable=yes --track-origins=yes -s ./your_program
is memory leak?
Spreading your code over multiple languages means you now have a bunch new fun edge cases to deal with when things get stuck and don't update correctly. With everything in one language that means all debuggers and other tools work the same regardless of where in the app I'm debugging.
I've found it fascinating to see UI building come full circle. I started in code (C), then using various XML schemes and now back in code (Kotlin, Flutter, Swift). But a big part of why it's nice now is because C is a pretty horrid language for making modern applications.
this is completely unrelated to what QML is.
QML is a complete language which allows to build both the application logic and the layout, which extends javascript with native reactive data binding. QML+QtQuick is equivalent to Dart+Flutter, or JS+HTML+CSS+React for instance.
https://www.qt.io/product/qt6/qml-book/ch04-qmlstart-qml-syn...