You are viewing information archived from Mozilla.org on 2015-08-05.
EXPOSING HTML/DOM VIA ATK/AT-SPI
Contributors
Will Walker, Sun Microsystems
Aaron Leventhal, IBM
Cathy Laws, IBM
Larry Weiss, IBM
Pete Brunet, IBM
OVERVIEW
This document specifies 2 different ways that document
content can be efficiently exposed using the existing ATK/AT-SPI
without
requiring any modifications to the ATK/AT-SPI specifications. It
does so by effectively creating a containment hierarchy based upon a
relevant subset of the DOM, where each container has an accessible
role for the
associated element type (e.g., "html:h1", "html:h2", "html:a", html:p",
etc.). Contiguous regions of text appear as children that are
accessible text objects, and other components (e.g., widgets inside
forms) appear as their associated ATK/AT-SPI counterparts. Furthermore,
CSS
attributes will be represented as AT-SPI text
attributes.
PROBLEM STATEMENT
Screen reader users need the ability to:
Navigate documents by higher level structure
Know where they are within the current document's structure
Although requirement 1 can be filled by an improved caret navigation
capability, that would force synchronization of the screen reader and
the document's point of regard, which is not always possible -- for
example, the Firefox
caret cannot be set within a list of options, for example. The caret
browsing system is difficult to maintain because the two can be out of
sync.
Furthermore, requirement 2 really means that the screen reader needs to
be able to determine document structure anyway.
The current ATK implementation in Gecko has the following
problems, when exposing document structure:
It does not expose headings, quotations, paragraphs, forms,
list containers and other structural objects
It is not possible to differentiate sequential or nested
lists from each other or even from a single large list. The list items
are simply exposed one after the next.
There is confusion about embedded objects. They are
currently exposed out of order relative to where they actually exist in
the document. Gecko must do extra error-prone work to expose them in
the new order, and the AT must unravel that. In addition, the extra
work in the Gecko implementation makes code reuse with the Windows/MSAA
implementation difficult.
There is confusion about links. In HTML links can span
entire paragraphs or a complex set of objects. This is not modeled well
with the current system.
PROPOSAL
The proposal is relatively simple:
Continue to map HTML constructs to
existing AT-SPI roles where natural from both the Gecko and AT-SPI
perspectives.
Expose new roles for
structural HTML element. The
rolenames will be of the form "HTML:tag", where 'tag' is the actual
HTML
tag.
Treat container elements as containers which can have children. Embedded objects, links and other containers will appear as
children of these
containers.
Expose text with one of 2 methods:
a) expose text as child text leaf nodes with a role of "text", or
b) Expose contatenated
text on the container for the text, using unicode 0xfffc to indicate embedded objects
A general rule is: if something exposing AccessibleText has children, then each
child will be marked in the text with 0xfffc
Treate image maps as a container image with a set of anchor
children
Hypertext will still be necessary, but only to return a Hyperlink object for getting
the URL of a link
The following interfaces would be supported: AccessibleAction,
AccessibleComponent, AccessibleText, AccessibleEditableText,
AccessibleImage, AccessibleSelection, AccessibleTable, AccessibleValue, AccessibleRelation
AccessibleTable would be implemented on the <table> object,
and the children could also be obtained by walking the regular
hierarchy.
NEW ROLES
Here are the new roles to be exposed:
Markup
Semantic meaning
ATK/AT-SPI role
h1...h6
headings
html:h1 ... html:h6
ul, ol, li
lists and list items
html:ul, html:ol
abbr, acronym
abbreviations & acronyms
html:abbr and html:acronym
dl, dd, dt
definition lists
html:dl, html:dd, html:dt
form
forms
html:form
blockquote, q
quotations
html:blockquote, html:q
tbody, tfoot, thead
table grouping
html:tbody, html:thead, html:tfoot
div and all other text containers styled with display:block
All items that can be interacted with via the keyboard will support
the FOCUSABLE,
FOCUSED, ENABLED and SENSITIVE states, and will emit "focus:" events when
they receive focus. All items that display text will
implement Accessibility_Text. Furthermore, "object:text-caret-moved"
events should be emitted by the associated text object as the caret
moves.
NOTE: We discussed the potential overlap between "html:ul",
"html:ol", and the AT-SPI "list" role. We decided
the AT-SPI
"list" role
is a good map to lists inside HTML forms, but "html:ul" and "html:ol"
merit their own roles. A similar situation exists
for HTML tables,
but we have not discussed this yet.
NOTE: The pattern of "html:tag" can also be carried over to
other XML specifications, such as SVG, MATHML, ODF, etc. The
general pattern is "namespaceabbrev:tag".
ADVANTAGES
Document structure exposed. Screen reader can get info it
needs.
Simplfies implementation both for Gecko and for screen
readers
Backwards-compatible with GOK
Synchronizes ATK
implementation with MSAA implementation, making an
implementation for advanced text interfaces on Windows much easier down the road,
by allowing reuse of all the classes which implement special interfaces
(text, tables, etc.). With this huge difference removed, most of the
important code to implement those interfaces can be moved into the
cross-platform implenentation.
Q & A
Q. How is the checkable state of a menuitem, treeitem or listitem exposed?
A. There is a role_check_menu_item or
role_radio_menu_item. List items could contain checkboxes. Trees could
contain checkboxes and RELATION_NODE_CHILD_OF.
Q. How do I support the valuechange event? It is
too difficult to find the old value.
A. The spec doesn't actually say what detail1 & detail2 are for the
valuechange event. The AT does not need them. It gets everything it
needs from AccessibleValue, so disregard those detail1 and detail2 for
valuechange events.
Q; How do I expose the indeterminate state of a progressbar?
A. Return
an error when for the value getter in AccessibleValue.
Q. How can an AT like a magnifier scroll to a particular piece of text?
A.
Set the caret before the word you want to scroll to
Q. How should an application indicate that an alert, which will not receive focus,
has become visible?
A. Use the children-changed event to indicate when
an alert becomes visible or a state-change event for the visible state. Unresolved: what's visible-data-changed?
Q. How do we expose the visited state for a link?
A. For now you need to use ATK_STATE_CHECKED, but in the future we hope to add ATK_STATE_TRAVERSED
to the API (matching MSAA)
Q. Should an assistive technology use role indentifiers or role strings?
A. The
current role extension mechanism of ATK works OK for
string-based rolenames. When it comes to role identifiers (i.e., the
enumerated role type), it does not appear to permit various AT-SPI
implementations to cooperate well in a distributed environment - there
appears to be the possibility that identical role
identifiers could exist for different rolename strings. To
resolve this problem, it is advisable to ignore numeric role
identifiers and instead use rolename strings.
ISSUES TO BE RESOLVED
This proposal currently does not yet provide a way to expose tag names for all
elements, DOM attribute/value pairs, computed style attribute/value
pairs associated with individual elements, such as the size of
a table cell's border. Do we need to add a getProperty(name) method to the Accessibility_Accessible
interface? Do we need an Accessibility_DOMNode interface?
Need to document how text selection is exposed
Show pseudo code for walking into children
Document how AtkHypertext interface will be used on an html:a object
Should the namespace come after the tagname and a separator? This would be more extensible to DHTML role names.
This proposal does not deal with multiple roles from DHTML. Thus
far, the answer has been that ATK uses multiple interfaces to do the
same thing, but that does not map well to DHTML accessibility.
Should a separate role or text attribute be maintained for text which is inserted
by the rendering environment, such as list bullet text, :before/:after
styled text? In MSAA the separate "static text" role is used.
Should we remove the "html:" prefix from common roles that we'll want
to reuse for ODF, such as "h1"?
Positional information is not exposed for tree items, menu items,
list items, radio buttons or tab panels. GetIndexInParent() does not
always provide the right information, such as "Level x, Item n of m,
with c children" for a tree item. Gathering the information is too slow
for large collections of items, such as an a large list of email
messages. Trees have to be multilevel in ATK. Can we implement GetIndexInParent
in ATK so that it's not O(n)
* NEED TO ASK FOR ADDITION TO ATK/ATSPI * It is not currently possible to get the URL, mime type or doctype for a document.
EXAMPLES
The following examples provide a mapping between HTML and the ATK representation.
In each of the following examples, an ATK accessible object is encapsulated
in braces ("{}"), and meaningful ATK interfaces and attributes,
including specializations, are represented as name/value pairs inside the
braces. For convenience, accessible text is shown merely as 'text="contents
of the text"'.
To do: HTML table & DHTML role example
Note: everything supports AtkObject and AtkComponent
HTML content
HTML source
ATK
with text in leaves
ATK
with text in containers
This is a heading
This is a paragraph with an image in it.
This is another heading
<h1>This is a heading</h1> <p> This is a paragraph with an <image src="image.gif" alt="some image"/> image in it. </p> <h2>This is another heading</h2>
{parent role="html:h1"} {child AtkText, role=ATK_ROLE_TEXT, text="This is a heading"} {parent role="html:p"} {child AtkText, role=ATK_ROLE_TEXT, text="This is a paragraph with an"} {child AtkImage, role=ATK_ROLE_IMAGE, AccName/ImageDescription ="some image"} {child AtkText, role=ATK_ROLE_TEXT, text="image in it."} {parent role="html:h2"} {child AtkText, role=ATK_ROLE_TEXT, text="This is another heading"}
{parent AtkText, role="html:h1", text="This is a heading"} {parent AtkText, role="html:p", text="This is a paragraph with an ? image in it"} {child AtkImage, role=ATK_ROLE_IMAGE, AccName/ImageDescription ="some image"} {parent AtkText, role="html:h2", text="This is another heading"}
<ul> <li>This is a list item.</li> <li>This is another list item.</li> </ul>
{parent role="html:ul"} {child role="html:li"} {grandchild AtkText, role=ATK_ROLE_TEXT, text=<<<bullet>>>} {grandchild AtkText, role=ATK_ROLE_TEXT, text="This is a list item."} {child role="html:li"} {grandchild AtkText, role=ATK_ROLE_TEXT, text=<<<bullet>>>} {grandchild AtkText, role=ATK_ROLE_TEXT, text="This is another list item."}
{parent AtkText, role="html:ul", text="??"} {child AtkText, role="html:li", text="<<<bullet>>> This is a list item."} {child AtkText, role="html:li", text="<<<bullet>>> This is another list item."}
This is a list item.
This is another list item.
<ol> <li>This is a list item.</li> <li>This is another list item.</li> </ol>
{parent role="html:ol"} {child role="html:li"} {grandchild AtkText, role=ATK_ROLE_TEXT, text="1."} {grandchild AtkText, role=ATK_ROLE_TEXT, text="This is a list item."} {child role="html:li"} {grandchild AtkText, role=ATK_ROLE_TEXT, text="2."} {grandchild AtkText, role=ATK_ROLE_TEXT, text="This is another list item."}
{parent AtkText, role="html:ol", text="??"} {child AtkText, role="html:li", text="1. This is a list item."} {child AtkText, role="html:li", text="2. This is another list item."}
This is a list item.
Nested item 1.
Nested item 2
This is another list item.
<ul> <li> This is a list item. <ul> <li>Nested item 1</li> <li>Nested item 2</li> </ul> </li> <li>This is another list item.</li> </ul>
{parent role="html:ul"} {child role="html:li"} {grandchild AtkText, role=ATK_ROLE_TEXT, text=<<<bullet>>>} {grandchild AtkText, role=ATK_ROLE_TEXT, text="This is a list item."} {grandchild role="html:ul"} {great-grandchild role="html:li"} {great-great-grandchild AtkText, role=ATK_ROLE_TEXT, text=<<<bullet>>>} {great-great-grandchild AtkText, role=ATK_ROLE_TEXT, text="Nested item 1"} {great-grandchild role="html:li"} {great-great-grandchild AtkText, role=ATK_ROLE_TEXT, text=<<<bullet>>>} {great-great-grandchild AtkText, role=ATK_ROLE_TEXT, text="Nested item 2"} {child role="html:li"} {grandchild AtkText, role=ATK_ROLE_TEXT, text=<<<bullet>>>} {grandchild AtkText, role=ATK_ROLE_TEXT, text="This is another list item."}
{parent AtkText, role="html:ul", text="??"} {child AtkText, role="html:li", text="<<<bullet>>> This is a list item.?"} {grandchild AtkText, role="html:ul", text="??"} {great-grand-child AtkText, role="html:li", text="<<<bullet>>> Nested item 1"} {great-grand-child AtkText, role="html:li", text="<<<bullet>>> Nested item 2"}
{child AtkText, role="html:li", text="<<<bullet>>> This is another list item."}
<form> <div> <label for="self"/> Tell me a little more: </label> </div> <div> <textarea> I am a monkey with a long tail. I like to swing from trees and eat bananas. I've recently taken up typing and plan to write my memoirs. </textarea> </div> </form>
{parent role="html:form"} {child role="html:div"} {grandchild AtkRelation, role=ATK_ROLE_LABEL, ATK_RELATION_LABEL_FOR} {great-grandchild AtkText, role=ATK_ROLE_TEXT, text/name="Tell me a little more:"} {child role="html:div"} {grandchild AtkEditableText, AtkRelation, AtkStateSet, AtkAction, or ATK_ROLE_ENTRY, text="I am a monkey with ..."}, ATK_RELATION_LABELLED_BY, STATE_MULTILINE,
STATE_REQUIRED allowed}
{parent AtkText, role="html:form", text="??"} {child AtkText, role="html:div", text="?"} {grandchild AtkText, role=ATK_ROLE_LABEL, ATK_RELATION_LABEL_FOR, text="Tell me a little more:"} {child role="html:div"} {grandchild AtkEditableText, AtkRelation, AtkStateSet, AtkAction, ATK_ROLE_ENTRY, text="I am a monkey with ..."}, ATK_RELATION_LABELLED_BY, STATE_MULTILINE, STATE_REQUIRED allowed}