MathML Core

BlinkOn 11 - November, 2019
Sunnyvale, California

{\Gamma(t)} = {\int_{0}^{+\infty} x^{t-1} e^{-x} dx} = {\frac{1}{t} \prod_{n=1}^\infty \frac{\left(1+\frac{1}{n}\right)^t}{1+\frac{t}{n}} } \sim {\sqrt{\frac{2\pi}{t}} \left(\frac{t}{e}\right)^t}

Rob Buis (rbuis@)
Frédéric Wang (fwang@)

Preface: Igalia

A Parisian in good Spanish company

In a nutshell

View of the Tower of Hercules (A Coruña) and the surrounding area during sunrise.

A Coruña, Galicia
Worker-owned company
~90 people
Open Source/Standards
Browsers

Web Engines Hackfest

Chromium

~30 active contributors (abhijeet, alexander, antonio, antía, caitlin, cathie, dape, fred, gyuyoung, henrique, ivan, jacobo, javi, jessica, joanie, joyee, julie, loren, maksim, mario, martin nick, oriol, rego, rob, sergio, shin, simon)
Main areas
- Accessibility
- JavaScript/V8
- Layout (CSS, MathML)
- OnionSoup/s13n/CodeHealth
- Ozone/Linux (Wayland, X11)
- Scrolling

Web Standard
Committees

ARIA WG (joanie)
CSS WG (brian, javi, oriol, rego)
MathML CG (brian, fred, rob)
TC39 (dan)

Chapter I: Project

Plans of a prodemocratic group in NW Spain

Fundraising

Small donations via a non-profit

Implementation Roadmap

$Screenshot of chromium-review issues with a mathml tag$

Basic setup ✅
Basic layout ✅
Operator Dictionary ✅
Stretchy operators: ✅
Advanced style ✅
HTML5 Compatibility ●
Upstreamed & shipped ●

Launching Features

Idea Phase: explainer, intent-to-implement ✅
Design Phase: design doc & spec, TAG review, chromestatus entry ✅
Implementation Phase: implementation & tests, intent-to-ship, 3 LGTMs ●
Post Launch: crashes, regressions, bug fixes, interop, doc, cleanup... ❌

there is a standard process for the Chromium project to launch new features. Because of the very special situation of MathML (spec/implementation/controversies) and of the new LayoutNG implemented by Google's layout team, had to proceed a bit differently.
had to rework the spec and talk to different actors to reach a consensus ; worked on a branch to obtain concrete results for discussions, design ; contributed to interop effort with many tests. solid enough proposal.
actually already had most of what is needed before announcement on blink-dev. Last month completed the idea/design phase and started implementation.

Project Workflow

Chapter II: Interop

Frederic's dramatic changes

MathML Core

$Screenshot of mathml core spec$

Fundamental subset
- Ask CG members
- Analyze tools & use statistics
Implementation details
- TeX/OpenType layout
- HTML5/CSS compatibility
Browser-driven
- Gecko/WebKit/Blink
- Web Platform Tests!
Extensible
- MathMLElement IDL
- CSS/Houdini
- New MathML elements

First step: extract a subset of manageable size, corresponding to what is used in practice and implemented in browsers.
Enhance it with detailed rules => compatible with other web platform technologies and provides good math rendering. In particular one of our main effort has been to provide CSS-compatible layout.
Use experience and feedback from browsers to ensure the features integrate well in their code base. In addition to the new Chromium implementation, we've also done a bit of work on Gecko/WebKit to ensure better alignment with the new MathML Core spec.
difficult to reach consensus on what should be in core ; also concerns that removing MathML3 features could break backward compatibility. MathML Refresh CG decided to follow an extensible web approach: ensure web developers can implement features removed from MathML3 or more generally implement their own math extensions. Redirect to Brian

Web Platform Tests

~2000 tests (1, 2, 3)
- Math layout
- Removed features
- DOM/CSS interaction
Results (4)
- Igalia's Blink: ~99%
- Gecko: ~73%
- WebKit: ~72%

Besides the specification, important achievement has been to write automated tests. Currently ~2000 but still writing more.
3 categories: those checking the math layout, those checking that removed MathML3 features are not supported and finally all the tests related to interaction with the rest of the web platform.
Current results is pretty good. Our prototype for Blink as almost 100% and Gecko/WebKit about 3/4. remaining quarter essentially related to all new CSS, arguably not currently important for math users but likely to gain interest with the extensibility idea I previously mentioned, so we should fix them.

Past implementations

Firefox/Gecko 2008
Chrome/WebKit 2013
Opera/Presto 2007

2019 implementations

Firefox/Gecko July release
Epiphany/WebKit Build r249360
Chromium/Blink Igalia's Branch

Chapter III: Chromium

Tumultuous story of a navigator

Changesets

Base Setup

Runtime flag
mathml.css
Element classes
New CSS
New DOM/IDL

Low-level API

Font parameters
Glyph stretching
Layout constraints
Ink metrics

MathML

Dictionary
Attributes
Painters
Layout utils
Layout classes

Base setup (1/2)

Runtime Enabled Feature ✅
```
  name: "MathMLCore",
  status: "test",
```
Legacy support ✅
- virtual test suite
- mathml-fallback.css
MathML Core tests and expectations ✅
MathMLElement class ✅
MathMLElement DOM/IDL ✅
- *EventHandlers
- ElementCSSInlineStyle
- HTMLOrForeignElement

Base setup (2/2)

CSS tweaks ●
- mathml.css (e.g. default font-family)
- StyleAdjuster (e.g. display: contents;)
mathvariant
$x^{t - 1} e^{- x}$ , $x^{t - 1} e^{- x}$ , $x^{t - 1} e^{- x}$ , $x^{t - 1} e^{- x}$
displaystyle and scriptlevel
${\Gamma(t)} = {\frac{1}{t} \prod_{n=1}^\infty \frac{\left(1+\frac{1}{n}\right)^t}{1+\frac{t}{n}} }$ , $\frac{1}{t} \frac{1}{n}$
display: math; (algo determined from tag name)

will introduce some CSS tweaks, for example to use a math font-family by default, disable properties etc
by default math identifier with 1 letter automatically rendered as italic. OpenType math fonts use special Unicode code points for math italic characters, we proposed to introduce a new text-transform value for that purpose.
more Unicode characters & transforms can be specified by explicit mathvariant. Although one can directly use the Unicode characters, we thought it could be useful to add new text-transform values for them too.
notice that the font-size scaled down when we enter scripts, which corresponds to scriptlevel attribute. more subtilities e.g. font-size is preserved when entering a fraction but scaled down when entering entering another inner fraction.
This is controlled by another MathML attribute called displaystyle (true at level 0, false when entering level 1). other consequences e.g. notice that it changes the spacing: the font-size remains > 1/2 the original size but gap between num and bar is < 1/2 the original gap.
Finally the math algo determined from MathML tag name but proposed a new "display: math" value so that users can override it with existing or custom layout.

Low-level API (1/2)

Layout/Glyph constants: $\sqrt{\frac{2\pi}{t}}$ $\sqrt{\frac{2\pi}{t}}$
FontPlatformData::GetHarfBuzzFace
Size variants and glyph stretching:
$\int \int \prod \prod \sqrt{} \sqrt{} \sqrt{} ((($
GraphicsContext::DrawText, passing a NGTextFragmentPaintInfo with a custom ShapeResultView/ShapeResult

Low-level API (2/2)

Layout constraints ●: $(\frac{t}{e})$ $(\frac{t}{e})$
Parent, Stretchy, Non-Stretchy
Target stretch sizes for NGConstraintSpace
Ink metrics:
- Text box: (TextMetrics::actualBoundingBoxAscent/Descent)
- All boxes: ink-over/ink-under baselines

Layout of stretchy operators is performed as follows: you have the parent in red, with stretchy children in blue and non-stretchy children in green. The parent first layout the non-stretchy children to obtain a target stretch size.
Then it lays out the stretchy children, passing that target size. For that purpose, we had to introduce new layout spaces in NGConstraintSpace.
If you use the normal boxes of text, you'll generally get more spacing than needed. So another important aspect of math layout is to actually use exact ink metrics enclosing the text.
We can use ink metrics for text box only like in WebKit or implement something more general and distinguish standard and ink metrics for all boxes like Gecko.
The core spec follows Gecko but modern LaTeX engines seem to follow WebKit, which makes things simpler. We were not able to get any hint from Microsoft so far.

MathML (1/4)

Group similar elements
Create/update layout objects
Parse/update attributes

MathML (2/4)

Map inherited style attributes ●
- Existing ( dir, mathsize, mathcolor...)
- Internal ( displaystyle, mathvariant, scriptlevel...)
Map length attributes
- CSS length-%
- 5 internal slots
Parse enumerated attributes (boolean attributes and operator's form attribute prefix/infix/postfix)
Dictionary for default operator attribute values
- Key: (operator text, form)
- Values: Lengths lspace, rspace and boolean flags
- Optimization: Redundancy of values

Inherited style attributes will be mapped to existing style properties or the new ones I mentioned.
Non-inherited length attributes will be stored in shared internal CSS length-% properties. The highest number of length attributes per element is 5.
There are enumerated attributes that we can either just parse by custom methods on the element classes or map them using new internal properties.
Finally, some operator attributes take some value from a dictionary. The key is essentially the operator text and form. The values are lengths are boolean flags. Currently this is not really optimized but there are many redundancy so we expect we can reduce the size of the dictionary and maybe introduce caching.

MathML (3/4)

Better classes with painting?
Rewrite operator stretching?
Use TableNG?
Derive from LayoutNGMixin<LayoutBlock>?

MathML (4/4)

Use composition instead of inheritance
ComputeMinMaxSize()/Layout() as per the spec
Baselines, layout constraints, out-of-flow children, padding/border/margin...

Size Impact (2019-11-05)

		Δsize
Linux Desktop	`chrome.stripped`	+284KB
	`deb` archive	+66KB
	`rpm` archive	+42KB
Android	Native Code	+82KB
Android	Install	+85KB

	ΔLines
`core/layout`	+4823
`core/mathml`	+2876
`core/css`, `core/style`	+1339
`core/paint`	+298
`platform/fonts`, `platform/graphics`	+268
`core/dom`, `core/html`, `core/svg`	+56
Total	+9660

Open Issues

MathML Refresh CG (MathML simplification, CSS clarifications, layout improvements, better test coverage)
LayoutNG
- Tables
- Legacy layout forced (SVG, print, multicol, editing)
Accessibility
- ATK
- NSAccessibility
- MSAA / UI Automation?
Extensibility
- Custom elements/layout
- Expose more magic

Appendix: Demos

¿Questions?

crbug.com/6606

¡ 溜溜溜 !

Chromium logo is licensed under a Creative Commons Attribution 2.5 and BSD licenses.
TC39 logo licensed under CC0 ; W3C logo under the W3C Logo and Icon Usage.
Copyright of NISO, Pearson and APS logos belong to the corresponding companies, used with permission.
Copyright of Ernest Hemingway's book covers belong to the corresponding publishers or cover artists.
Use as a parody qualifies as fair use under the Copyright law of the United States.

The HTML slides are currently intended to be rendered with Igalia's Chromium build 2019/11/05, Latin Modern Math and STIX Two.

MathML Core

Preface: Igalia

In a nutshell

Web Engines Hackfest

Chromium

Web StandardCommittees

Chapter I: Project

Fundraising

Implementation Roadmap

Launching Features

Project Workflow

Chapter II: Interop

MathML Core

Web Platform Tests

Past implementations

2019 implementations

Chapter III: Chromium

Changesets

Base setup (1/2)

Base setup (2/2)

Low-level API (1/2)

Low-level API (2/2)

MathML (1/4)

MathML (2/4)

MathML (3/4)

MathML (4/4)

Size Impact (2019-11-05)

Open Issues

Appendix: Demos

¿Questions?

Web Standard
Committees