Since last week, pretty much the most requested feature has been text flow around inline graphics. Yes, even more than editability. I’ve had cleaning up and adding advanced features to the TextLayout and TextContainer on the tinytlf 1.0 roadmap for a while, but last night I finally got to work on it. These classes are only preliminary, but I hope they demo just how powerful tinytlf’s layout architecture can be.

As always, the source is available here: source for these demos.

Text Layout

Ok, so say we have this wikipedia entry about the fascinating Atrophaneura hector (Crimson Rose) butterfly. It’s a nice article, and tinytlf formats it well (except for the TLMR bug):

Don’t encyclopedia entries come with an image?

Much better!

Put that image where you want it

Alright, now we’re rockin’

Ok, I know this is ugly, but I thought I’d show off a little bit. You aren’t constricted to docking on the left or the right, the new layout algorithm will wrap text around images no matter where they are in the markup.

Features

This shows off some features I’ve never talked about before. Of course there’s flow around the image, but that’s really just some fancy layout math, it’s not too complicated. I’m probably most proud of the fact that tinytlf intelligently renders only the invalid TextLines.

Invalidation

This is a Flash Text Engine feature, but it’s one that I love: when members of the FTE ContentElement model change (text, ElementFormat, etc.), the TextBlock will tag the TextLines which render the content “invalid.” The FTE can’t automatically update the TextLines; whomever renders the TextLines (tinytlf, in this case), is responsible for surgically removing and re-rendering the invalid lines.

It’s a delicate procedure, but tinytlf handles it like a champ. You see the result of this in the examples whenever you roll over an anchor tag and it changes fontPosture or color.

Layout

The second part of this is the little bit of fancy math I did to break and layout the lines in the proper order. If you want to see the algorithm, check out the newest ImageFlowContainer here.

It’s not too difficult. Basically, as I lay out the lines, I calculate the (x, y) position for the next TextLine. Because I can change the x and y independently of each other, I can break TextLines across the plane of the graphic.

Where can it go from here? My next feature will be to respect padding set on the <img/> tag. After that will be allowing a way for the <img/> to specify whether it renders inline, causes line/paragraph breaking, etc. There’s a lot that can be done.

Caveats

I haven’t tested this with more than one image. In theory it should work, but I’ve been awake for longer than 24 hours, so I can’t trust I’m actually thinking as clearly as I think I am o.O.

And yes, there’s a bug with the links. It’s especially prominent here, but basically when you move the mouse very quickly, the FTE TextLineMirrorRegions dispatch a “mouseOver” but never its corresponding “mouseOut.” If anybody on Adobe’s TLF or FTE team can shed some light on this situation, I’d be very grateful.

That’s it, happy coding. Fork it on github!

Tags: Flash Text Engine, text layout, tinytlf

This entry was posted on Tuesday, August 24th, 2010 at 10:54 am and is filed under actionscript, community. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.

This is part 3 in an ongoing series about the Flash Text Engine. Here’s part 1 and part 2.

By now you should know this series isn’t about Adobe’s Text Layout Framework, which is an advanced typography and text layout framework. The Flash Text Engine is the low-level API that TLF is built on. In Flash Player 10, the FTE resides in the flash.text.engine package.

In FP 10 have this great new font rendering library, the Flash Text Engine, but it only fashions characters into TextLines of a specific width. When you think about everything else a TextField does, you begin to understand that rendering the glyphs is only a small (but still important!) percentage of the total work that’s done.

Text Layout

If you recall from my previous overview of the FTE, the main “Controller” class is TextBlock. TextBlock is a factory for TextLines. You supply a ContentElement to the TextBlock, it computes and creates the necessary TextLines to show that content. TextBlock’s rendering algorithms operate on a paragraph level; that is, one TextBlock per paragraph/one paragraph per TextBlock. For example, in the FTE, this paragraph should be represented as a single TextBlock, with multiple ContentElements that define formatting.

Paradigms from a superior layout engine

If you’re familiar with HTML styles, you know that HTML has two distinct layout paradigms: block layout and inline formatting. Block layout affects an entire block of text. Styles like padding, indentation, and margins affect layout on the block level. Inline formatting affects how the characters are rendered within blocks, such as color, size, posture, weight, justification, etc.

TextBlock’s algorithms take care of the inline formatting, because inline formatting affects whether characters flow between TextLines. It’s left up to you to apply any block formatting. For example, when you call the TextBlock’s createTextLine method, you choose the width of the TextLine. This simple option allows us, with some fancy math-e-matics, to achieve many properties of block level layout.

Layout Algorithms

We want some generalized methods for accomplishing various layouts. Everything from simple, single paragraph layouts, to layouts with block formatting, to complex mutli-TextBlock and multi-column (newspaper style) layouts.

I’ve already demonstrated the absolute simplest way: render all the lines in a while loop, finishing once the TextBlock returns null from createTextLine. This is easy and straightforward, and you can apply any kind of block formatting you wish.

var y:Number = 0;
var line:TextLine = block.createTextLine(null, 200);
while(line)
{
    addChild(line);
    y += line.ascent;
    line.y = y;
    y += line.descent;
    line = block.createTextLine(line, 200);
}

Source

Even though all it seems I’ve done is render a TextField, I’ve accomplished two tasks here: I’ve rendered every TextLine that the TextBlock decides I need, and, by incrementing a counter for the Y dimension, I’ve calculated a rudimentary layout for the TextLines.

Indentation

Now, applying indentation is super easy. Make the first line a little smaller, and change his x position to compensate.

var y:Number = 0;
var line:TextLine = block.createTextLine(null, 185);
line.x = 15;
while(line)
{
    addChild(line);
    y += line.ascent;
    line.y = y;
    y += line.descent;
    line = block.createTextLine(line, 200);
}

Source

Alignment

Center:

var y:Number = 0;
var line:TextLine = block.createTextLine(null, 200);
while(line)
{
    addChild(line);
    y += line.ascent;
    line.y = y;
    line.x = (200 - line.width) * 0.5;
    y += line.descent;
    line = block.createTextLine(line, 200);
}

Source

Right alignment is the same, only don’t multiply the calculated x by 0.5.

Source

See? Standard layout practices, ultimately the same math we use every day in component layouts. But, you say, “indentation and alignment are easy, it’s just calculating the x of the lines”. You’re right, it is easy. But so are the other block formatting properties like padding, margins, line spacing, etc. They’re all just calculating the correct x or y and conditionally applying them in the loop.

Multi-TextBlock layout

Ok, now we know how to layout lines from a single TextBlock. With a little code reuse, laying out multiple TextBlocks is a breeze:

var blocks:Vector.<TextBlock> = new <TextBlock>[block1, block2];
var y:Number = 0;
for(var i:int = 0; i < blocks.length; ++i)
{
    y = layoutBlock(blocks[i], y);
    y += 5;
}
// Returns the aggregate y after this layout operation
function layoutBlock(block:TextBlock, y:Number):Number
{
    var line:TextLine = block.createTextLine(null, 185);
    line.x = 15;
    while(line)
    {
        addChild(line);
        y += line.ascent;
        line.y = y;
        y += line.descent;
        line = block.createTextLine(line, 200);
    }
    return y;
}

Source

Multi-Container layout

Ah, now we’re getting to the good stuff. Multi-container layout is really cool, because it allows us to “overflow” text from one DisplayObjectContainer to another, which allows us, among other things, to achieve column layouts.

The general idea is to render as many TextLines into a DisplayObjectContainer (DOC) as possible. When we’ve hit his boundaries, switch to the next available DOC. We can accomplish this with a few modifications to the previous methods. The layout method needs to return the last TextLine that fit in the DOC. That way, we can re-enter the layout routine and pick up with the TextBlock where we left off.

var line:TextLine = layoutBlock(block, null, container1);
if(line)
    line = layoutBlock(block, line, container2);
 
// Returns the last line rendered out of the TextBlock
function layoutBlock(block:TextBlock, previousLine:TextLine, 
                             container:DisplayObjectContainer):TextLine
{
    var line:TextLine = block.createTextLine(previousLine, container.width);
    var y:Number = 0;
    while(line)
    {
        container.addChild(line);
        y += line.ascent;
        line.y = y;
        y += line.descent;
        //If we reached the height boundary, return the last line that fit.
        if(y + line.height > container.height)
            return line;
        line = block.createTextLine(line, container.width);
    }
    return line;
}

Source

Multi-TextBlock and Multi-Container layout

Here’s the really good stuff! Now we’re going to render multiple TextBlocks into multiple DisplayObjectContainers by merging the two methods above.

To solve this problem, lets identify our list of knowns:

1. We have a list of TextBlocks.
2. We have a list of DisplayObjectContainers to fit the TextBlocks into.
3. We wish to render as many lines into each DOC as possible.
4. When the DOC is full, switch to the next one and pick up where we left off.
5. We need to keep track of the last TextLine rendered, so we know where we left off.

From our previous experience with rendering multiple TextBlocks, we know that TextBlock will return null when he can render no more lines. And from our previous experience with rendering across DisplayObjectContainers, we know that when a Container is full, we should return the last line that fit. Therefore the logic plays out as such:

• Loop over each TextBlock.
• Render as many TextLines into the DOC as possible, returning the last line rendered.
  1. If the TextLine is null, we know the TextBlock ran out of lines and there’s more space in the DOC. Keep the same DOC, but move to the next TextBlock.
  2. If the TextLine is not null, there are still more lines in the TextBlock, but this DOC ran out of space. Keep the same TextBlock, but move to the next DOC.
• If we reach the end of either list, return.

var blocks:Vector.<TextBlock> = new <TextBlock>[block1, block2, block3, block4];
var containers:Vector.<DisplayObjectContainer> = 
    new <DisplayObjectContainer>[container1, container2, container3];
 
layout(blocks, containers);
 
function layout(blocks:Vector.<TextBlock>, containers:Vector.<DisplayObjectContainer>):void
{
    var blockIndex:int = 0;
    var containerIndex:int = 0;
 
    var block:TextBlock;
    var container:DisplayObjectContainer;
 
    var line:TextLine;
    while(blockIndex < blocks.length)
    {
        block = blocks[blockIndex];
        container = containers[containerIndex];
 
        line = layoutInContainer(container, block, line);
 
        if(line && ++containerIndex < containers.length)
        {
            container = containers[containerIndex];
            containerY = 0;
        }
        else if(++blockIndex < blocks.length)
            block = blocks[blockIndex];
        else
            return;
    }
}
 
var containerY:Number = 0;
 
function layoutInContainer(container:DisplayObjectContainer, 
                                   block:TextBlock, previousLine:TextLine):TextLine
{
    var line:TextLine = createTextLine(block, previousLine);
    while(line)
    {
        container.addChild(line);
        containerY += line.ascent;
        line.y = containerY;
        containerY += line.descent;
 
        if(containerY + line.height > container.height)
            return line;
 
        line = createTextLine(block, line);
    }
 
    //This will be null.
    return line;
}
 
function createTextLine(block:TextBlock, previousLine:TextLine):TextLine
{
    var w:Number = 190;
    var x:Number = 0;
    //Apply indention properties here.
    if(previousLine == null)
    {
        w -= 15;
        x += 15;
    }
    var line:TextLine = block.createTextLine(previousLine, w, 0.0, true);
    if(line)
        line.x = x;
    return line;
}

Source
Text Source

Woah! Take a breather

It’s a lot to digest, I know. But now you can see that TextLayout isn’t black-magic voodoo, and is entirely achievable in the new Flash Text Engine. If you have any questions, feel free to comment or email me. If you have any techniques for text layout, or have any comments on my techniques, I’d love to hear those too. I took some shortcuts with these demos, but I’ve built out more complete and performance-tuned layouts into tinytlf, the small text layout framework I’ve been working on clandestinely for a few months.

Aside: Text Layout vs. Component Layout

In typical component based layout engines (such as Flex’s), child creation is separate from layout. Usually all the children are added to the display list first, then laid out at some other time. Children aren’t created or destroyed based on their positions or sizes on the screen, and layout doesn’t affect the creation of future children (this is assuming we’re not talking about virtualized layout, which is a special case).

Working with the sizing and layout of TextLines, the block-level layout properties (like padding, indentation, etc.) dictate how each TextLine is created and laid out. This in turn affects how the TextBlock renders the next TextLine, and so on and so forth. I haven’t been able to separate block-level layout properties from the TextLine creation process. This isn’t so bad in practice, but sometimes it rubs me the wrong way, I feel like there should be a better way and I just haven’t found it yet.

Tags: Flash Text Engine, FTE, text layout, tinytlf

This entry was posted on Sunday, August 8th, 2010 at 7:25 pm and is filed under actionscript, community. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.

This is the first post in what will be a multi-part series about the Flash Text Engine, a new low level text API in Flash Player 10.

To clarify, this series isn’t about Adobe’s Text Layout Framework, which is an advanced typography and text layout framework. The Flash Text Engine is the low-level API that TLF is built on. In Flash Player 10, the FTE resides in the flash.text.engine package.

The FTE is designed to render text “document style”. It’s primarily meant to replace the TextField for advanced uses, not provide a whole framework for text layout on the scale of an HTML rendering engine.

The FTE handles what I call flow: formatting that causes text to be pushed to the next line in a paragraph. I don’t know if that’s the official term, but it seems to fit. It does not do layout, which is things like bullet points, indentation, wrapping around images, padding, etc., nor does it handle decoration, things like underline, strikethrough, background color, selection, etc. I believe the FTE leaves these out because 1.) layout is a much more complicated and nuanced problem than flow, one that you wouldn’t necessarily want in the FP core, and 2.) decorations don’t cause reflow, or affect whether and how text is wrapped to the next line.

The FTE conforms to a small MVC architecture, there are about 10 core classes that provide most of the functionality, with the rest of them encapsulating constants. Something to note, every class in the FTE is final :( more on that later.

The FTE Model

The basis of the Flash Text Engine model is something called ContentElement. ContentElement is an abstract base class. You never call new ContentElement() (it’s similar to DisplayObject in this regard), instead you instantiate one of its 3 subclasses: TextElement, GraphicElement, or GroupElement. Collectively these classes describe a Tree hierarchy for text, but I want to talk a bit more about ContentElement before we get too deep into that.

ContentElement

Take a look at the constructor of ContentElement:

ContentElement(elementFormat:ElementFormat = null, eventMirror:EventDispatcher = null, textRotation:String = "rotate0")

It has two important arguments, elementFormat and eventMirror (as well as a third less important argument, unless you’re one of the crazy types who likes to rotate text). I will come back to the eventMirror later, but for now lets just talk about ElementFormat.

The ElementFormat class describes most of the properties that handle text flow. It has a fontDescription member, which is exactly what it sounds like. In FontDescription you’ve got your standard fontFamily, fontWeight, fontPosture (which is traditionally the fontStyle in Flash), along with how the font is supposed to be retrieved from the depths of the Flash Player (as Compact Font Format or a device font).

ElementFormat has properties like alpha, color, baselineShift, kerning, etc. Basically anything that can affect reflow.

Ok, so that describes all you need to know for now about the ElementFormat and FontDescription objects. Now onto the implementation class you’ll use.

TextElement

Out of the three, TextElement is the most straightforward. It simply accepts a string of text to take care of:

TextElement(text:String = null, elementFormat:ElementFormat = null, eventMirror:EventDispatcher = null, textRotation:String = "rotate0")

The ElementFormat you pass in is applied to the entire string of text that this TextElement owns. So if you specify an ElementFormat with a color of red, the entire string of text will render red.

GraphicElement

The next one to worry about is GraphicElement. He accepts any DisplayObject instance (instance!), as well as the width and height that you wish to allocate for the Graphic in the text:

GraphicElement(graphic:DisplayObject = null, elementWidth:Number = 15.0, elementHeight:Number = 15.0, elementFormat:ElementFormat = null, eventMirror:EventDispatcher = null, textRotation:String = "rotate0")

Some of the properties of the ElementFormat will apply to the GraphicElement, such as alpha, baselineShift, etc. Obviously the GraphicElement doesn’t respect font-specific settings from the ElementFormat and FontDescription objects.

GroupElement

Lastly there’s the GroupElement:

GroupElement(elements:Vector.<ContentElement> = null, elementFormat:ElementFormat = null, eventMirror:EventDispatcher = null, textRotation:String = "rotate0")

GroupElement is critical. GroupElement is a collection of any combination of TextElements, GraphicElements, or other GroupElements. GroupElement is the Tree functionality of FTE’s model. TextElement can’t have children, it controls a single String. Likewise, GraphicElement only describes a single DisplayObject instance. GroupElements tie it all together.

GroupElements provide an API for doing standard Tree functions; you can retrieve, split, merge, and group children using various methods. I speak from experience when I say you won’t often mess with this unless you’re writing an editable text field. And if you are writing an editable text field, God help you (just kidding, it is hella fun).

OK, enough Model talk. Onwards to…

The FTE View

There are two (2!) classes that make up the entirety of the Flash Text Engine’s View division: TextLine and TextLineMirrorRegion. Right now you can forget about TextLineMirrorRegion, as that has to do with interaction, which is a complicated topic and one which I will cover in detail later. So for now, only focus on TextLine.

TextLine

TextLine is a DisplayObjectContainer. Yes, that means it has the get/add/removeChild methods (they still work!), and is also an InteractiveObject. You can listen for all the normal interaction events. However, even though it inherits from InteractiveObject, there are a few properties that you can only read, not write. Those are detailed in the documentation for TextLine.

TextLine adds the concept of atoms, which are indivisible characters in a TextLine. Individual characters are atoms, as well as any graphics you have. The important thing to know here is that atoms can never be split between lines. The FTE will measure only to the atom level, no lower.

Atom information can be expensive to keep around… At first the TextLine only renders its text, it doesn’t know anything about the atoms it contains. However calling various methods will cause the TextLine to create its atom data. For example, if you call getAtomIndexAtPoint(), the TextLine must create the info about each atom so it can then calculate which atom occurs at the point you specify. This is all well and good, but be sure to call flushAtomData() once you’re done so the atom data will be GC’d.

TextLine has a reference to the previous and next lines, because TextLine is also a doubly-linked list! How convenient! Of course, if there is no previous or next, you know you’re the first or last lines, respectively.

TextLine also has a validity status, which is whether the ContentElement that the line represents has changed since the line was rendered. Values are described in the TextLineValidity class.

One thing that TextLine definitely is not: a Sprite. No, TextLine is a DisplayObjectContainer. The most important implication from this is that TextLine has no graphics context. This means you can’t call textLine.graphics.draw. :( Oh well.

TextLine is a concrete class, you use it directly, but you cannot instantiate one by calling its constructor. To do that you need…

The FTE Controller

There is arguably one class in the FTE’s Controller division: TextBlock. I say arguably because yeah, TextJustifier and TabStop exist, but they just affect how TextBlock does its rendering, not… hm. Ok, I’ve convinced myself that they count as Controller classes too, but only barely.

But believe me, you will come to think of TextBlock as the only Controller class too.

The TextBlock is a fairly standard Factory pattern implementation: TextBlock’s primary job is to accept a ContentElement as input and output as many TextLines as you want, given a width. ContentElement -> TextBlock -> TextLines. Got it? Me neither.

Ok, so TextBlock has this method called createTextLine():

createTextLine(previousLine:TextLine = null, width:Number = 1000000, lineOffset:Number = 0.0, fitSomething:Boolean = false):TextLine

Ok so what you do is you pass in the previous line that you created, plus the width that you want the current line to be, and TextBlock will measure out a TextLine for you. Are you seeing the doubly-linked list yet?

If you want to create the first line from a TextBlock, you should just pass in null to the createTextLine() method; assuming the TextBlock has content in his content property, and that content has at least one atom (characters or graphics), passing in null will always return a TextLine. If there are no more lines to be created, TextBlock will return null from the call to createTextLine().

So from this it is simple to render the lines for a TextBlock with width 200:

var y:Number = 0;
var line:TextLine = block.createTextLine(null, 200);
while(line)
{
    addChild(line);
    y += line.height;
    line.height = y;
    line = block.createTextLine(line, 200);
}

Ok, I’ve detailed a lot so far, now it’s time to get to at an example.

Flash:

Here’s the code for the above simple line rendering:

package
{
  import flash.display.Sprite;
  import flash.text.engine.ContentElement;
  import flash.text.engine.ElementFormat;
  import flash.text.engine.FontDescription;
  import flash.text.engine.FontPosture;
  import flash.text.engine.FontWeight;
  import flash.text.engine.GroupElement;
  import flash.text.engine.TextBlock;
  import flash.text.engine.TextElement;
  import flash.text.engine.TextLine;
 
  [SWF(width="450", height="32")]
  public class SimpleDemo1 extends Sprite
  {
    public function SimpleDemo1()
    {
      super();
 
      var e1:TextElement = new TextElement('Consider, what makes a text line a ', new ElementFormat(new FontDescription(), 24));
      var e2:TextElement = new TextElement('text line', new ElementFormat(new FontDescription("_serif", FontWeight.NORMAL, FontPosture.ITALIC), 24));
      var e3:TextElement = new TextElement('?', new ElementFormat(new FontDescription(), 24));
 
      var e:Vector. = new Vector.();
      e.push(e1, e2, e3);
 
      var block:TextBlock = new TextBlock(new GroupElement(e));
      var line:TextLine = block.createTextLine(null, stage.stageWidth);
 
      var _y:Number = 0;
      while(line)
      {
        addChild(line);
        _y += line.height;
        line.y = _y;
        line = block.createTextLine(line, stage.stageWidth);
      }
    }
  }
}

Holy crap Batman!
As you can see, it required 3 different TextElements and a GroupElement to render some freakin’ italic text in the middle of a sentence. Yeah. Par for the frickin’ course.

In part 2 I’ll get into more details about interaction, TextBlock manipulation, all of it.
Till then watch this project on github: tinytlf. It’s due for some major updates but it’s what I’m going to start talking about soon.

Tags: advanced text rendering, cff, compact font format, flash, flash player 10, Flash Text Engine, FTE, text layout, Text Layout Framework, TextField, tiny tlf, tinytlf, TLF

This entry was posted on Thursday, June 3rd, 2010 at 4:18 am and is filed under actionscript, community. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.