Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Overhaul Text Arrangement and Trimming #1556

Open
wants to merge 6 commits into
base: develop
from

Conversation

@VisualMelon
Copy link
Contributor

VisualMelon commented May 10, 2020

Fixes #1531, #1559, and #1554

Checklist

  • I have included examples or tests
  • I have updated the change log (will do so if/when this is striped back and becomes a set of meaningful commits)
  • I am listed in the CONTRIBUTORS file
  • I have cleaned up the commit history (use rebase and squash)

Changes proposed in this pull request:

  • Adds the ITextMeasurer interface, and implementations for all RenderContexts that can support it
  • Adds the ITextTrimmer interface, and SimpleTextTrimmer implementation which uses 'character ellipsis' trimming mode as its default
  • Adds the TextArranger class, which arranges multiline text as per #1554 and #1551
  • Changes all platforms in this repo to use TextArranger for text arrangement (thereby resolving #1531 and #1559 )
  • Adds an SvgExporter to OxyPlot.ImageSharp with additional tests
  • Removes the UseVerticalAlignmentWorkaround property from OxyPlot.Svg as it is redundant and will only get in the way

Big Issues

  • API for ITextTrimmer is weak: the render context determines the type of trimming that is done, which may not be ideal

@oxyplot/admins

@VisualMelon
Copy link
Contributor Author

VisualMelon commented May 12, 2020

MaxSize support is probably pretty slow, and will be caught out by strings with combining characters (need to do some research on that), but gives pretty reasonable results. It could be performed more quickly by render contexts with access to glyph information (e.g. SkiaSharp), where cropping the line would be logarithmic (i.e. irrelevant) rather that log-linear.

WPF (the bottom-most line is drawn clipped)

image

Skia on WPF (the bottom-most line is skipped completely)

image

@VisualMelon
Copy link
Contributor Author

VisualMelon commented May 12, 2020

@Jonarw would you mind taking a gander at the changes in Skia, and let me know if this looks too abhorrent to you to be an acceptable change?

Having done the provisional implementation of maxSize, I think putting all this stuff in one place is a good idea, but it may be inappropriate to clip by text rather than pixels for e.g. SkiaSharp if it can do clipping like occurs with WPF. That might motivate adding additional functionality to ArrangeText so that it can arrange according to maxSize but not perform the clipping itself.

@Jonarw
Copy link
Member

Jonarw commented May 12, 2020

Generally I think the ArrangeText() function is a very good idea to make text rendering more consistent between the platforms. However I think we should think a bit more about what exactly this should do.

I am a bit sceptical if clipping on character level is really the way to go for implementing maxSize. I think this could be unreliable for certain character combinations, and performance will probably fairly bad (though this should not be our main concern).

I am more concerned that we might be trying to do something very complex that we don't actually really need (see also my comment in #1559) - after all, I haven't missed this feature on SkiaSharp so far.

@Jonarw
Copy link
Member

Jonarw commented May 12, 2020

I am also not too sure how about I feel about guessing the font metrics (ascent, descent, leading) by doing multiple text measurements. At the current state, the exact algorithm for text height measurement should be an implementation detail of the render context, which we shouldn't rely on. If we effectively move the responsibility for vertical text arrangement out of the render context and into the ArrangeText() function, we should also adapt the API accordingly to make the responsibilities clear. This could mean:

  • IRenderContext.MeasureText() explicitly only works for single lines of text and only measures the width
  • IRenderContext.DrawText() explicitly only works for single lines of text
  • IRenderContext offers some way to get the relevant font metrics (ascent, descent, leading) the ArrageText() function needs to do its job properly
@VisualMelon
Copy link
Contributor Author

VisualMelon commented May 12, 2020

@Jonarw yeah, it's pretty horrendous, and I think what you describe is actually close to how OxyPlot was designed: DrawText is even documented as not working with multi-line text.

To that end, I would suggest keeping the current API as it is (i.e. MeasureText measures whole texts, DrawText should support multi-line text), but add additional methods which provide what you describe (i.e. measure width of text, retrieve metrics for a given font/size/weight combination). It could be that these are not made available through IRenderContext - because axes and series and what-not shouldn't need them - but through another ITextMeasurer interface. That would avoid any breaking changes, and make it easy to push all the text-arranging stuff into it's own class. (This new class could provide methods for computing the multi-text widths/heights, so all the render context has to do is implement the basic features you describe).

@Jonarw
Copy link
Member

Jonarw commented May 12, 2020

You are right of course, from the perspective of Axes, Series etc. the IRenderContext interface should not change.
Or maybe it could change, and we replace the removed/changed APIs by extensions, similar to the RenderingExtensions.DrawClippedXXX() stuff?

I haven't completely wrapped my mind around this, I need to think about it some more...

@VisualMelon
Copy link
Contributor Author

VisualMelon commented May 12, 2020

Thanks for the comments on what must look a complete mess at the moment. I'll probably next implement your suggestions about abstracting out the 'basic' measurement stuff (so we don't have to infer metrics), and put the whole thing in its own class. Rough plan:

  • add ITextMeasurer interface with methods
    • GetFontMetrics(family, size, weight) returning ascender, descender, and leading (probably in a struct if I can be bothered to add another new file)
    • Measure(text, family, size, weight) returning the width of the text
  • add TextArranger class with members
    • ITextMeasurer Measurer (or would we rather pass it as a parameter as I have been doing so far? All DPI awareness is in the ITextMeasurer: should the RenderContext have to know about this?)
    • MeasureText(text, family, size, weight) retuning the size of the (potentially multiline) text as an OxySize according to jonarw's proposal in #1551
    • ArrangeText(...) doing what it does now, but leaning on the ITextMeasurer to arrange (potentiall multiline) text however it is asked to do so
  • patch ImageSharp, SkiaSharp, and managed-only SVG RenderContexts as necessary

The additional interface will duly provide the information the managed only SvgRenderContext needs to do probably text aligning because it won't have to guess the descent anymore.

@VisualMelon VisualMelon force-pushed the VisualMelon:SvgVAlignAgain branch from f6ced01 to e5c9a80 May 13, 2020
@VisualMelon VisualMelon changed the title Svg Text Alignment Overhaul Text Arrangement and Trimming May 13, 2020
@VisualMelon VisualMelon force-pushed the VisualMelon:SvgVAlignAgain branch from e5c9a80 to 5eb86c3 May 13, 2020
@VisualMelon
Copy link
Contributor Author

VisualMelon commented May 13, 2020

Recent changes:

  • Moved ArrangeText into it's own class
  • Moved text-trimming into its own class which implements ellipsises
  • Broke Winforms somewhat (can't seem to get GetFontMetrics right, any input welcome)
  • Moved Baseline offset into ArrangeText, so the DrawText methods are even tidier than ever
  • Squashed everything to give the illusion that I know what I'm doing

Next tasks:

  • Fix winforms
  • Refit RenderContexts so that they lean on TextArranger.MeasureText and move the measuring logic into MeasureTextWidth (so they don't get into an infinite loop)

Any feedback (esspecially negative) welcome as always.

@VisualMelon
Copy link
Contributor Author

VisualMelon commented May 14, 2020

Refitting done, so ImageRenderContext, SvgRenderContext, and SkiaRenderContext shouldn't change any more. @Jonarw all else aside, does the interface to TextArranger look half-sane?
We could go further and extract ITextWriter from the render contexts: this would make sense for the various SvgExporter overrides, as currently there is unpleasantness associated with using render contexts that don't do any rendering, and the TextArranger has for each render context realistically has to be created in the constructor.

Also refitted the TextTrimmer so it supports Unicode and adding additional tests. This of course has made it slower... but I don't think that should be a concern: it is rarely used, and would be providing a capability that is presently unavailable almost everywhere. I doubt it is perfect (i.e. there will be strings that it gives nonsense results on), but it should be fairly easy to extend by making the central regexes more complicated. Annoyingly we can't compile the regexes because we started .NET Standard 1.0; however, the architecture means that if someone really cared they can swap out the text trimmer for their own; whether this is a good architecture is another issue.

WinForms is still broken: either I'm having difficultly extracting the font metrics (i.e. Graphics.RenderContext.GetFontMetrics is broken), or DrawString is doing something unexpected.

@VisualMelon
Copy link
Contributor Author

VisualMelon commented May 14, 2020

Fixed WinForms.

I think that means everything is working as I understand it should. I've not StyleCop'd this, but I'm going to open for review because there is nothing else I'm intending to do at the moment.

@VisualMelon VisualMelon marked this pull request as ready for review May 14, 2020
@VisualMelon VisualMelon force-pushed the VisualMelon:SvgVAlignAgain branch 2 times, most recently from 8a10280 to 1927d3d May 14, 2020
@Jonarw
Copy link
Member

Jonarw commented May 14, 2020

Looks very promising!
I did not get into this too deep, but the general concept looks sensible to me. Over the weekend I will have a closer look.

Just for my understanding: This adapts ImageSharp, SkiaSharp, WinForms and Pdf, at the moment, right? So WPF would still have to be adjusted for this (but could also be its own issue and PR).

Also we should then add more sophisticated ITextTrimmers where possible (i.e. for SkiaSharp) - but again, this does not have to be part of this PR in my opinion.

@VisualMelon VisualMelon force-pushed the VisualMelon:SvgVAlignAgain branch from 1bd167b to f6aa96c May 14, 2020
@VisualMelon
Copy link
Contributor Author

VisualMelon commented May 14, 2020

@Jonarw thanks. I'll eagerly await your feedback, and perhaps put my efforts into #1516

It also adapts the built-in SVG renderer, which now depends on an ITextMeasurer rather than a whole IRenderTarget, which is something that could be exploited in the future.

I've not refitted WPF yet because frankly I'm frightened of it, but I can do that work for this PR or it can be done later if @objorke approves. For the purposes of consistent vertical alignment it is probably a good idea, but it may still be that we leave the trimming to it (though as I've discussed elsewhere, trimming by character seems a much better strategy to me) which could complicate matters with multi-line text

Regards advanced trimmers: yep, that was pretty much what I had in mind. The performance so far is perfectly good in my opinion, but there is plenty of scope for improvement. I won't claim the current trimmer behaviour is perfect, but it should be fairly easy to adapt if we find difficulties, and the architecture means it should be easy to support users without regular releases.

@VisualMelon VisualMelon force-pushed the VisualMelon:SvgVAlignAgain branch 3 times, most recently from 255f460 to 44867b0 May 16, 2020
@Jonarw
Copy link
Member

Jonarw commented May 21, 2020

To me this looks very good now!
@VisualMelon is this ready from your side?
@objorke would you be ok with this?

@VisualMelon VisualMelon force-pushed the VisualMelon:SvgVAlignAgain branch from 44867b0 to 77ad229 May 21, 2020
@VisualMelon
Copy link
Contributor Author

VisualMelon commented May 21, 2020

(Rebased)

I don't believe that there was anything more I intended to do with this.

Considerations for the future, perhaps:

  • Review text trimming API (should it be configurable through IRenderContext?)
  • More tests for the text trimmer(s), preferablly written by someone who knows a little more about unicode than I.
  • Should TextArranger be a member of RenderContextBase? (this would also help with consistency.)
  • Should the text measuring/font-handling code be pulled out of the render contexts? (This would be beneficial in a couple of places (e.g. making the various SVG exporters lighter), but may just be adding complexity.
  • A text trimmer that can use glyph information (e.g. as is available to Skia) for more efficient text trimming (this will be a concern for any change in a API, as glyph may not map to character (I don't know enough to really comment.))
  • MathText integration
@Jonarw
Copy link
Member

Jonarw commented Aug 5, 2020

I just stumbled over this PR again and I still think it would be very nice to merge this!

I think after all this time we can only assume that there would be no objections from objorke's side.

@VisualMelon Would you rebase this to the current develop, so we could verify this with the latest changes?

@VisualMelon
Copy link
Contributor Author

VisualMelon commented Aug 5, 2020

@Jonarw my main reservation is that it changes the text trimming (this could be moved to separate PR). I've not really solicited @objorke's feedback since this came to be in some sort of a stable state, so I've been keen to wait a little longer now that I've actually pinged him here.

I'll get on with rebasing and retesting anyhow, and I'd better make some new examples to show text trimming as it is intended. (they already exist)

@VisualMelon VisualMelon force-pushed the VisualMelon:SvgVAlignAgain branch from 77ad229 to b067a3a Aug 5, 2020
@objorke
Copy link
Member

objorke commented Aug 10, 2020

Sorry, I am not able to follow all the conversations here at the moment. But I am very happy to see all the improvements you are implementing on the library, it is way more thorough than the original design. I agree text trimming could be a separate PR, but wouldn't object if both are merged in this one! Keep going!
ps. I hope performance is not severely affected with these changes.

@VisualMelon
Copy link
Contributor Author

VisualMelon commented Aug 10, 2020

@objorke thanks for the comment.

The only performance implications should be when there are large numbers of text annotations/labels/etc. with a max size that actually constrains them, otherwise it's just a big refactoring that makes everything a little more consistent.

The trimming algorithm is O(n log(n)) where n is the number of characters in a line of text. @Jonarw and I have discussed options of replacing this with a linear algorithm on platforms that provide per-glyph character information, which can be done without modifying the core library under with this architecture, but I've not seen a situation where any significant number of plot elements need trimming, so absolute performance hasn't been a priority. I'll put together some excessive examples to see how they behave.

@VisualMelon
Copy link
Contributor Author

VisualMelon commented Aug 10, 2020

Added two performance examples with ~100 items of text: trimmed in one case, not trimmed in the other. With WinForms and the WPF/Skia renderer, the trimming example is observably slower, but not significantly so.

With WPF, it is significantly slower, which is a regressions as previously the trimming was just as fast. However, I can't observe any degradation in responsiveness if TextMeasurementMethod.GlyphTypeface is used, and both examples are noticeably quicker.

The cost is a function of the speed at which text is measured by the platform, and the TextMeasurementMethod.GlyphTypeface seems to be much faster than deferring to TextBlock. Is there a reason it is not used by default? Since it does a scan through the glyphs themselves, it is an obvious place we could use a linear trimming method in the future.

@VisualMelon VisualMelon mentioned this pull request Aug 17, 2020
2 of 4 tasks complete
@Jonarw
Copy link
Member

Jonarw commented Aug 24, 2020

I think the performance should be good enough for now. We can tweak this in the future for render contexts where we have individual glyph information and the situation with WPF should already be much better once we can go through #1609.

Copy link
Member

Jonarw left a comment

Just came across some small nitpick-stuff

Source/OxyPlot/Rendering/FontMetrics.cs Outdated Show resolved Hide resolved
Source/OxyPlot/Rendering/FontMetrics.cs Outdated Show resolved Hide resolved
Source/OxyPlot/Rendering/TextArranger.cs Outdated Show resolved Hide resolved
Source/OxyPlot.Tests/Rendering/MockTextMeasurer.cs Outdated Show resolved Hide resolved
@VisualMelon
Copy link
Contributor Author

VisualMelon commented Aug 24, 2020

@Jonarw thanks. I'd better let StyleCop have another go at it soon.

/// <summary>
/// Gets or sets a value indicating whether to squash the bounds of trimmed text to the size the text.
/// </summary>
public bool SquashTrimmedTextBounds { get; set; }
Comment on lines 44 to 47

This comment has been minimized.

@Jonarw

Jonarw Aug 24, 2020 Member

I can't quite figure out what exactly this property does - could this be explained a bit more clearly?

This comment has been minimized.

@VisualMelon

VisualMelon Aug 29, 2020 Author Contributor

I'm struggling to explain this clearly... maybe a picture will help for your benefit (obviously the documentation needs text).

This is SquashTrimmedTextBounds = true:
squash

This is SquashTrimmedTextBounds = false:
nosquash

Note how the text is squashed toward it's alignment dot with true.

When true, if text is trimmed horizontally, then the bounds are squashed to the bounds of trimmed value; otherwise they are 'clamped' to the max size. This only changes the way that the text is arranged, and looking at those two figures now, I think it should be true. We could remove it and just treat it as true... that would solve all my problems

This comment has been minimized.

@VisualMelon

VisualMelon Aug 29, 2020 Author Contributor

Just realised this only applies vertically... the horizontal is already bounded... I think I'll make it 'hard coded' true instead, to reduce confusion.

@VisualMelon VisualMelon force-pushed the VisualMelon:SvgVAlignAgain branch from c786bc1 to 68d194c Aug 29, 2020
@VisualMelon
Copy link
Contributor Author

VisualMelon commented Aug 29, 2020

(Need to fix the tests after #1645)

@VisualMelon
Copy link
Contributor Author

VisualMelon commented Aug 29, 2020

Thinking about #391, do we need to be able to measure the size of trimmed text? E.g. if something asks for 3 lines of text, but only has space for 2.5 lines, we may not want it allocating the space for 2.5 lines when it could allocate space for just 2.

@Jonarw
Copy link
Member

Jonarw commented Aug 30, 2020

Thinking about #391, do we need to be able to measure the size of trimmed text? E.g. if something asks for 3 lines of text, but only has space for 2.5 lines, we may not want it allocating the space for 2.5 lines when it could allocate space for just 2.

This may make sense. But to be consistent, this should then also apply in the horizontal direction, i.e. it should return the actual width (including ellipsis character) and not the maxWidth of trimmed text.

@Jonarw Jonarw mentioned this pull request Sep 19, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

3 participants
You can’t perform that action at this time.