The Flash Text Engine, Part 2: Interaction
by Paul Taylor, Jun. 28, 2010, under [ actionscript, community ]

This is part 2 in an ongoing series about the Flash Text Engine. You can read part 1 here.

To clarify, this series isn’t about Adobe’s Text Layout Framework, which is an advanced typography and text layout framework. The Flash Text Engine is the low-level API that TLF is built on. In Flash Player 10, the FTE resides in the flash.text.engine package.

Interaction in the Flash Text Engine

In my previous post on the Flash Text Engine, I ran through the basics of what you need to get the FTE to render TextLines. While rendering lines on the screen is nice, this post is about how to add interaction to the TextLines that are produced.

TextLines are InteractiveObjects. You can add event listeners directly to them and listen for interaction events. The FTE also gives you the option to associate an individual EventDispatcher instance with a single ContentElement, so that when the user interacts with the data of the ContentElement, the events are cloned to the EventDispatcher instance you specified. As I discuss the details, you’ll see that each approach has its own strengths and weaknesses.

Approach 1: TextLines as InteractiveObjects

Since TextLine is an InteractiveObject, you can simply listen for Mouse and Keyboard events on each TextLine instance. With this approach, you know the TextLine that was interacted with. The main drawback here is that TextLine knows almost nothing about the ContentElement which it is rendering. Multiple ContentElements can be rendered into the same TextLine, and multiple TextLines can render the same (really long) ContentElement.

Interact with the lines in this demo:
TextLine InteractiveObject event listeners.
Source

The fact that you don’t know about the content of the TextLines is ok though, for some problems that isn’t necessary. For example, you don’t really need to know about the contents of the TextLines to draw decorations, such as underline, strikethrough, or selection.

Select this text:
Selection with the Flash Text Engine
Source

Approach 2: Working with TextLineMirrorRegions (TLMRs)

The preferred method of managing interaction in the Flash Text Engine is with TextLineMirrorRegions.

If you read my previous post, you’ll remember that to render any text, you have to create instances of any of the Flash Text Engine’s model classes: TextElement, GraphicElement, or GroupElement. When you create an instance of these classes, you can specify an EventDispatcher as the eventMirror for the ContentElement. When the user interacts with the visual representation of this ContentElement via TextLines, the events are re-dispatched to the eventMirror you specified. This allows you to know when a user interacts only with a particular ContentElement.

In this code sample, I create an EventDispatcher to pass in as the eventMirror for the TextElement. Then I add a listener for mouseMove on the eventMirror instance. This will trace out every time you mouse over the TextElement.

var dispatcher:EventDispatcher = new EventDispatcher();
new TextElement('Inspiring quote here.', new ElementFormat(
                                         new FontDescription()), 
                                         dispatcher);
var onMouseMove:Function = function(e:MouseEvent):void{
    trace('Mouse move on ' + e.target.toString());
}
dispatcher.addEventListener(MouseEvent.MOUSE_MOVE, onMouseMove);

These two lines are part of the same TextElement:
Quick demo of using the eventMirror with a TextElement.
Source

How is this different from the previous demo? TextLine has a property called mirrorRegions, a Vector of TextLineMirrorRegion instances. Since multiple ContentElements can be rendered by a single TextLine, TextLine creates TLMR instances for each ContentElement with an eventMirror, then associates the TLMRs with the eventMirrors respectively.

TextLine listens on itself for interaction events. When the events overlap with any of the TLMRs, TextLine notifies the appropriate TLMR of the event. After all normal event processing for the TextLine is done, each TLMR re-dispatches the events it was notified of to its eventMirror instance.

In this example, I added a listener for the “mouseDown” event on both the TextLine and the ContentElement’s eventMirror. Notice that the event dispatched on the eventMirror happens second.
The difference between adding an event listener on the line, and one via an eventMirror in FTE.
Source

Here’s what the TLMRs look like (I’ve drawn boxes for each boundary of a TextLineMirrorRegion).
Example of the bounds of TextLineMirrorRegions.
Source

Caveats

Of course, this wouldn’t be a Flash Player feature if it didn’t come with caveats ;).

TextLineMirrorRegion simulates the events, it doesn’t re-dispatch the exact instance it received from the TextLine. This is because TLMR isn’t an InteractiveObject itself. If you utilize the eventMirror to listen for MouseEvents, just realize they’re all faked — even though TextLine is the target, they didn’t originate from TextLine, and they don’t have feelings like real player-native events do.

rollOver/rollOut events

This event simulation means that we’re at the mercy of what Adobe chooses to simulate. They didn’t feel the need to simulate the roll events (rollOver/rollOut), so if you try to listen for them on the eventMirror, you won’t get them. Presumably this is because the roll events aren’t needed; since ContentElements don’t have display-list children, the roll events would be exactly the same as mouseOver/mouseOut.

Except the roll events are still very relevent.

It’s true, we’ve shifted from a display-list hierarchical structure (DisplayObjectContainers, etc.) to a ContentElement hierarchical structure. And it’s true, ContentElements don’t have display-list children. But they can have other ContentElement children, which means the roll events are still very relevant.

For example, if you had this XML model to render:

<p>
  Outside the group. 
  <group>
    <text color="#44AA00">
      First group child.
    </text>
    <text color="#AA0044">
      Second group child.
    </text>
  </group>
  Outside the group.
</p>

You might want to know only when the entire group node is interacted with (just like when you have a DisplayObjectContainer with children).

Here’s the demo of this model. Mouse between the boundary of the first child and the second child, and notice how you get a “mouseOut” and then another “mouseOver” from the group. If this were the roll events, you would only get the “mouseOut” and “mouseOver” from the children, but hear nothing from the group. FYI, “mouseDown” clears the debug lines. Why the rollOver/rollOut events aren't obsolete in the FTE.
Source

Comparison

So, how do the two techniques match up? The short answer is that each one accomplishes a different task. If you need the very base of interaction capabilities without the context of what you’re messing with (e.g. text selection), adding listeners straight on your TextLines is the way to go. However, if you need the context of which ContentElement the user interacts with (e.g. to mimic an HTML anchor tag), there’s no way around it, you have to use the event mirroring approach.

P.S. Isn’t it freaky interacting with text that you can’t select? Maybe I’m just OCD, but I feel a strong desire to see an I-Beam mouse cursor every time I hover over FTE text. My favorite demo to write was the second one, because not only did I get to come up with a quick selection-drawing method, I added the freakin’ I-Beam cursor. Anyway, hope you enjoyed this and good luck :).

Tags: , , , ,

The Flash Text Engine, Part 1: Overview
by Paul Taylor, Jun. 3, 2010, under [ actionscript, community ]

This is the first post in what will be a multi-part series about the Flash Text Engine, a new low level text API in Flash Player 10.

To clarify, this series isn’t about Adobe’s Text Layout Framework, which is an advanced typography and text layout framework. The Flash Text Engine is the low-level API that TLF is built on. In Flash Player 10, the FTE resides in the flash.text.engine package.

The FTE is designed to render text “document style”. It’s primarily meant to replace the TextField for advanced uses, not provide a whole framework for text layout on the scale of an HTML rendering engine.

The FTE handles what I call flow: formatting that causes text to be pushed to the next line in a paragraph. I don’t know if that’s the official term, but it seems to fit. It does not do layout, which is things like bullet points, indentation, wrapping around images, padding, etc., nor does it handle decoration, things like underline, strikethrough, background color, selection, etc. I believe the FTE leaves these out because 1.) layout is a much more complicated and nuanced problem than flow, one that you wouldn’t necessarily want in the FP core, and 2.) decorations don’t cause reflow, or affect whether and how text is wrapped to the next line.

The FTE conforms to a small MVC architecture, there are about 10 core classes that provide most of the functionality, with the rest of them encapsulating constants. Something to note, every class in the FTE is final :( more on that later.

The FTE Model

The basis of the Flash Text Engine model is something called ContentElement. ContentElement is an abstract base class. You never call new ContentElement() (it’s similar to DisplayObject in this regard), instead you instantiate one of its 3 subclasses: TextElement, GraphicElement, or GroupElement. Collectively these classes describe a Tree hierarchy for text, but I want to talk a bit more about ContentElement before we get too deep into that.

ContentElement

Take a look at the constructor of ContentElement:

ContentElement(elementFormat:ElementFormat = null, eventMirror:EventDispatcher = null, textRotation:String = "rotate0")

It has two important arguments, elementFormat and eventMirror (as well as a third less important argument, unless you’re one of the crazy types who likes to rotate text). I will come back to the eventMirror later, but for now lets just talk about ElementFormat.

The ElementFormat class describes most of the properties that handle text flow. It has a fontDescription member, which is exactly what it sounds like. In FontDescription you’ve got your standard fontFamily, fontWeight, fontPosture (which is traditionally the fontStyle in Flash), along with how the font is supposed to be retrieved from the depths of the Flash Player (as Compact Font Format or a device font).

ElementFormat has properties like alpha, color, baselineShift, kerning, etc. Basically anything that can affect reflow.

Ok, so that describes all you need to know for now about the ElementFormat and FontDescription objects. Now onto the implementation class you’ll use.

TextElement

Out of the three, TextElement is the most straightforward. It simply accepts a string of text to take care of:

TextElement(text:String = null, elementFormat:ElementFormat = null, eventMirror:EventDispatcher = null, textRotation:String = "rotate0")

The ElementFormat you pass in is applied to the entire string of text that this TextElement owns. So if you specify an ElementFormat with a color of red, the entire string of text will render red.

GraphicElement

The next one to worry about is GraphicElement. He accepts any DisplayObject instance (instance!), as well as the width and height that you wish to allocate for the Graphic in the text:

GraphicElement(graphic:DisplayObject = null, elementWidth:Number = 15.0, elementHeight:Number = 15.0, elementFormat:ElementFormat = null, eventMirror:EventDispatcher = null, textRotation:String = "rotate0")

Some of the properties of the ElementFormat will apply to the GraphicElement, such as alpha, baselineShift, etc. Obviously the GraphicElement doesn’t respect font-specific settings from the ElementFormat and FontDescription objects.

GroupElement

Lastly there’s the GroupElement:

GroupElement(elements:Vector.<ContentElement> = null, elementFormat:ElementFormat = null, eventMirror:EventDispatcher = null, textRotation:String = "rotate0")

GroupElement is critical. GroupElement is a collection of any combination of TextElements, GraphicElements, or other GroupElements. GroupElement is the Tree functionality of FTE’s model. TextElement can’t have children, it controls a single String. Likewise, GraphicElement only describes a single DisplayObject instance. GroupElements tie it all together.

GroupElements provide an API for doing standard Tree functions; you can retrieve, split, merge, and group children using various methods. I speak from experience when I say you won’t often mess with this unless you’re writing an editable text field. And if you are writing an editable text field, God help you (just kidding, it is hella fun).

OK, enough Model talk. Onwards to…

The FTE View

There are two (2!) classes that make up the entirety of the Flash Text Engine’s View division: TextLine and TextLineMirrorRegion. Right now you can forget about TextLineMirrorRegion, as that has to do with interaction, which is a complicated topic and one which I will cover in detail later. So for now, only focus on TextLine.

TextLine

TextLine is a DisplayObjectContainer. Yes, that means it has the get/add/removeChild methods (they still work!), and is also an InteractiveObject. You can listen for all the normal interaction events. However, even though it inherits from InteractiveObject, there are a few properties that you can only read, not write. Those are detailed in the documentation for TextLine.

TextLine adds the concept of atoms, which are indivisible characters in a TextLine. Individual characters are atoms, as well as any graphics you have. The important thing to know here is that atoms can never be split between lines. The FTE will measure only to the atom level, no lower.

Atom information can be expensive to keep around… At first the TextLine only renders its text, it doesn’t know anything about the atoms it contains. However calling various methods will cause the TextLine to create its atom data. For example, if you call getAtomIndexAtPoint(), the TextLine must create the info about each atom so it can then calculate which atom occurs at the point you specify. This is all well and good, but be sure to call flushAtomData() once you’re done so the atom data will be GC’d.

TextLine has a reference to the previous and next lines, because TextLine is also a doubly-linked list! How convenient! Of course, if there is no previous or next, you know you’re the first or last lines, respectively.

TextLine also has a validity status, which is whether the ContentElement that the line represents has changed since the line was rendered. Values are described in the TextLineValidity class.

One thing that TextLine definitely is not: a Sprite. No, TextLine is a DisplayObjectContainer. The most important implication from this is that TextLine has no graphics context. This means you can’t call textLine.graphics.draw. :( Oh well.

TextLine is a concrete class, you use it directly, but you cannot instantiate one by calling its constructor. To do that you need…

The FTE Controller

There is arguably one class in the FTE’s Controller division: TextBlock. I say arguably because yeah, TextJustifier and TabStop exist, but they just affect how TextBlock does its rendering, not… hm. Ok, I’ve convinced myself that they count as Controller classes too, but only barely.

But believe me, you will come to think of TextBlock as the only Controller class too.

The TextBlock is a fairly standard Factory pattern implementation: TextBlock’s primary job is to accept a ContentElement as input and output as many TextLines as you want, given a width. ContentElement -> TextBlock -> TextLines. Got it? Me neither.

Ok, so TextBlock has this method called createTextLine():

createTextLine(previousLine:TextLine = null, width:Number = 1000000, lineOffset:Number = 0.0, fitSomething:Boolean = false):TextLine

Ok so what you do is you pass in the previous line that you created, plus the width that you want the current line to be, and TextBlock will measure out a TextLine for you. Are you seeing the doubly-linked list yet?

If you want to create the first line from a TextBlock, you should just pass in null to the createTextLine() method; assuming the TextBlock has content in his content property, and that content has at least one atom (characters or graphics), passing in null will always return a TextLine. If there are no more lines to be created, TextBlock will return null from the call to createTextLine().

So from this it is simple to render the lines for a TextBlock with width 200:

var y:Number = 0;
var line:TextLine = block.createTextLine(null, 200);
while(line)
{
    addChild(line);
    y += line.height;
    line.height = y;
    line = block.createTextLine(line, 200);
}

Ok, I’ve detailed a lot so far, now it’s time to get to at an example.

Flash:

The Flash plugin is required to view this object.

Here’s the code for the above simple line rendering:

package
{
  import flash.display.Sprite;
  import flash.text.engine.ContentElement;
  import flash.text.engine.ElementFormat;
  import flash.text.engine.FontDescription;
  import flash.text.engine.FontPosture;
  import flash.text.engine.FontWeight;
  import flash.text.engine.GroupElement;
  import flash.text.engine.TextBlock;
  import flash.text.engine.TextElement;
  import flash.text.engine.TextLine;
 
  [SWF(width="450", height="32")]
  public class SimpleDemo1 extends Sprite
  {
    public function SimpleDemo1()
    {
      super();
 
      var e1:TextElement = new TextElement('Consider, what makes a text line a ', new ElementFormat(new FontDescription(), 24));
      var e2:TextElement = new TextElement('text line', new ElementFormat(new FontDescription("_serif", FontWeight.NORMAL, FontPosture.ITALIC), 24));
      var e3:TextElement = new TextElement('?', new ElementFormat(new FontDescription(), 24));
 
      var e:Vector. = new Vector.();
      e.push(e1, e2, e3);
 
      var block:TextBlock = new TextBlock(new GroupElement(e));
      var line:TextLine = block.createTextLine(null, stage.stageWidth);
 
      var _y:Number = 0;
      while(line)
      {
        addChild(line);
        _y += line.height;
        line.y = _y;
        line = block.createTextLine(line, stage.stageWidth);
      }
    }
  }
}

Holy crap Batman!
As you can see, it required 3 different TextElements and a GroupElement to render some freakin’ italic text in the middle of a sentence. Yeah. Par for the frickin’ course.

In part 2 I’ll get into more details about interaction, TextBlock manipulation, all of it.
Till then watch this project on github: tinytlf. It’s due for some major updates but it’s what I’m going to start talking about soon.

Tags: , , , , , , , , , , , ,