Skip to content

Drop the FT2Font intermediate buffer. #30059

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: text-overhaul
Choose a base branch
from

Conversation

anntzer
Copy link
Contributor

@anntzer anntzer commented May 16, 2025

Directly render FT glyphs to the Agg buffer. In particular, this naturally provides, with no extra work, subpixel positioning of glyphs (closing #29551) (this could also have been implemented in the old framework, but would have required careful tracking of subpixel offets).

Note that rendering of mathtext boxes (e.g., fraction bars) is currently missing, but should be "easy" (the main question being whether/how to ensure proper hinting/alignment to pixel edges in the horizontal or vertical output cases). Likewise, all baseline images should be regenerated. Finally, the new APIs added to FT2Font are likely up to bikeshedding (but they are all private).

In practice this should get included (if we agree to go this way) into the FreeType update (#29816). This would also supersede the patch advertised at #29816 (comment).

test modified from #29551 to also include mathtext:

import matplotlib.pyplot as plt
import matplotlib.animation as manim

fig = plt.figure(figsize=(2, 2))
ts = [
    fig.text(0, 0, '$ABC123$'),
    fig.text(0, 0.2, 'ABC123'),
]
change = 0.001

def update(*args, **kwargs):
    global change
    for t in ts:
        x, y = t.get_position()
        if not (0 <= x <= 1 and 0 <= y <= 1):
            change *= -1
        t.set_x(x + change)
        t.set_y(y + change)
    return ts

ani = manim.FuncAnimation(fig, update, interval=20, frames=int(2/change),
                          cache_frame_data=False)
ani.save('text.gif')

PR summary

PR checklist

@story645
Copy link
Member

story645 commented May 16, 2025

Would you be open to using a tracking issue or project board for the font work (sample categories: draft, waiting on other PRs, ready for review, please review first)?

I think breaking everything up into small PRs is fantastic but I'm finding it a bit overwhelming to keep track of the order for reviewing things (and figure I'm not alone here) and how these PRs relate to each other.

@anntzer
Copy link
Contributor Author

anntzer commented May 16, 2025

Sure, I'll do that. I'm sorry for the mess, but this is basically me trying to remember all the patches or even just ideas I have accumulated over the years that went nowhere because they would thrash all the baseline images, and that I now try to present given that I see an opportunity with the FreeType upgrade. Also, some of them are not fully complete patches but things that I believe can be made complete with relatively little effort (for some value of "little effort"...) simply because I do not have the time now to finalize them but want to put them up for discussion as the underlying idea should be clear.

Edit: see the wiki; attn @QuLogic as well too.

@story645
Copy link
Member

story645 commented May 16, 2025

I'm sorry for the mess, but

No apologies needed, I commend you for taking advantage of the freetype upgrade. And thanks for the wiki, though some reason can't link cleanly ☹️

@anntzer
Copy link
Contributor Author

anntzer commented May 16, 2025

link fixed

@anntzer anntzer force-pushed the ft-direct-render branch 3 times, most recently from d44360f to 66fdae0 Compare May 17, 2025 09:55
@anntzer
Copy link
Contributor Author

anntzer commented May 17, 2025

Edit: fraction bar rendering has been added too.

@QuLogic
Copy link
Member

QuLogic commented May 22, 2025

A few warnings from compiling locally:

[2/4] Compiling C++ object src/ft2font.cpython-313-x86_64-linux-gnu.so.p/ft2font_wrapper.cpp.o
../../src/ft2font_wrapper.cpp: In lambda function:
../../src/ft2font_wrapper.cpp:1785:22: warning: unused variable ‘error’ [-Wunused-variable]
 1785 |             if (auto error = FT_Load_Glyph(face, idx, static_cast<FT_Int32>(flags))) {
      |                      ^~~~~
../../src/ft2font_wrapper.cpp:1788:22: warning: unused variable ‘error’ [-Wunused-variable]
 1788 |             if (auto error = FT_Render_Glyph(face->glyph, FT_RENDER_MODE_NORMAL)) {
      |                      ^~~~~
../../src/ft2font_wrapper.cpp: In lambda function:
../../src/ft2font_wrapper.cpp:1804:26: warning: unused variable ‘error’ [-Wunused-variable]
 1804 |                 if (auto error = FT_Glyph_To_Bitmap(&g, FT_RENDER_MODE_NORMAL, &origin, 1)) {
      |                          ^~~~~

@anntzer
Copy link
Contributor Author

anntzer commented May 22, 2025

This is semi-intentional: if these checks fail (error > 0) then the error should really be raised using throw_ft_error, not with a plain raise as I currently do, but throw_ft_error is not visible from ft2font_wrapper.cpp so some code needs to be moved around, e.g. make throw_ft_error an inline function defined in ft2font.h, or move the new implementations into real C++ FT2Font methods in ft2font.cpp and add wrapper lambdas in ft2font_wrapper.cpp, or (would be my preferred choice) switch error checking to the FT_CHECK macro used in mplcairo (see mplcairo's macros.h).
I left the warning as is just to keep this as a side discussion point, but it can trivially be suppressed by not capturing the error value too.

@QuLogic
Copy link
Member

QuLogic commented May 24, 2025

On a side note, I'm in favour of this in general as removing the intermediate buffer will make emoji/colour glyphs easier since they won't need to go through it.

@QuLogic
Copy link
Member

QuLogic commented May 30, 2025

On the call, we were discussing whether it made sense to target all these font-related (and requiring full image rebuild) PRs to a separate feature branch, which would contain the final test image updates. Do you have any thoughts on retargetting this PR (instead of folding it in to #29816)?

@anntzer
Copy link
Contributor Author

anntzer commented May 30, 2025

On the call, we were discussing whether it made sense to target all these font-related (and requiring full image rebuild) PRs to a separate feature branch, which would contain the final test image updates. Do you have any thoughts on retargetting this PR (instead of folding it in to #29816)?

Sure, I don't really mind either way? Should the branch live on this repo and be ultimately pruned out, or on yours?

@anntzer anntzer force-pushed the ft-direct-render branch from e980d97 to 383e594 Compare May 30, 2025 08:37
@anntzer
Copy link
Contributor Author

anntzer commented May 30, 2025

Fixed #30059 (comment) by using FT_CHECK (thus also needs #30123).

@QuLogic QuLogic changed the base branch from main to text-overhaul June 5, 2025 01:47
@QuLogic QuLogic added this to the v3.11.0 milestone Jun 5, 2025
@QuLogic QuLogic moved this to Ready for Review in Font and text overhaul Jun 5, 2025
self._renderer.draw_text_image(
bitmap["buffer"],
bitmap["left"],
int(self.height) - bitmap["top"] + bitmap["buffer"].shape[0],
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just checking whether we should round here or not?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think self.height is already always a integer-valued float, as it's the size of the buffer in pixels (I have a bit of doubt about hiDPI though). In fact, it would probably be nicer if self.height (and likewise self.width) was made an int (in fact, get_width_height, which is a FigureCanvasBase API, unlike self.{width,height}, which are specific to the Agg canvas, returns ints).

np.full((round(h), round(w)), np.uint8(0xff)),
round(x + dx), round(y - dy - h),
0, gc)
else:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This latter half is seemingly untested.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess we don't test rotated mathtext fractions? Something like figtext(.5, .5, r"$\frac{3}{4}$", rotation=45) would do.

Comment on lines 1777 to 1780
// TODO: Return a nicer structure than dicts.
// NOTE: The lifetime of the buffers is limited and could get invalidated...
// TODO: Real antialiasing flag.
// TODO: throw_ft_error.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are these TODO nice-to-have, or to-be-done here? Also, throw_ft_error is done now?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed the mention of throw_ft_error.
We should definitely add support for antialiasing, but that should be easy, just a matter of threading the parameter down to the right place.
Returning a nicer structure than a plain dict would be nice but not really an obligation as long as _render_glyph stays private. Likewise, we can probably get away with not caring about the buffer lifetime as long as _render_glyph is private, but properly handling the buffer lifetime once that becomes a public API seems trickier? I basically wrote the minimum API that works for privately communicating between the C++ and Python layers, but didn't put a lot of thought into it.

Comment on lines 1789 to 1790
auto matrix = FT_Matrix{
std::lround(c / hf), std::lround(-s), std::lround(s / hf), std::lround(c)};
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
auto matrix = FT_Matrix{
std::lround(c / hf), std::lround(-s), std::lround(s / hf), std::lround(c)};
auto matrix = FT_Matrix{
std::lround(c / hf), std::lround(-s), std::lround(s / hf), std::lround(c)};

or maybe, since it's 2D:

Suggested change
auto matrix = FT_Matrix{
std::lround(c / hf), std::lround(-s), std::lround(s / hf), std::lround(c)};
auto matrix = FT_Matrix{
std::lround(c / hf), std::lround(-s),
std::lround(s / hf), std::lround(c)};

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yup

Directly render FT glyphs to the Agg buffer.  In particular, this
naturally provides, with no extra work, subpixel positioning of glyphs
(which could also have been implemented in the old framework, but would
have required careful tracking of subpixel offets).

Note that all baseline images should be regenerated.  The new APIs added
to FT2Font are also up to bikeshedding (but they are all private).
@QuLogic
Copy link
Member

QuLogic commented Jun 10, 2025

Sorry for the conflict; I realize that #29199 was going to be thrown away here anyway, but I wanted to acknowledge the (new) contributor's work, and I knew you wouldn't have any trouble rebasing.

@anntzer
Copy link
Contributor Author

anntzer commented Jun 10, 2025

I didn't look carefully but I believe I can just get rid of the changes in #29199 when rebasing because my PR essentially already does the same thing, correct? (no problem with doing the rebase, of course)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: Ready for Review
Development

Successfully merging this pull request may close these issues.

3 participants