Mono Accessibility 2.1


Last Tuesday, we presented Mono Accessibility 2.1. We worked really hard on this release. Our main goals were, among other things, to improve our UI Automation Client API implementation, polish the interaction with at-spi2, better Moonlight accessibility and to handle custom and client-side providers. The great work made by all the contributors was the reason this release accomplished those goals.

What happened since last time?

Explaining this release is kind of hard if you are not involved in Mono Accessibility, and even more difficult if you’re not involved in Accessibility at all. However, if you’re familiar with Mono and .NET you are, also, aware of the possibility of developing .NET-based applications that run on both platforms with minor* changes or no changes at all. But, before the implementation of Mono Accessibility, there was something missing: all those applications were not accessible.

This meant two things. First Atk-based Assistive Technologies (ATs) were not able to access Windows Forms nor Silverlight applications. Second .NET-based ATs using UI Automation were not able to run on Mono. Making them useless for people requiring Accessibility to interact with software running Mono on Linux**.

After the implementation of Mono Accessibility, we have:

  • UI Automation API Framework
  • Bridge between UI Automation and ATK

This way ATs using either Accessibility Technology, Atk or UI Automation, will be able to interact with Gtk+, Windows Forms and Silverlight applications, with no changes at all.

This is what happened

In the following video: (if you are using a RSS reader click this link) gcalctool is launched and an application, using UI Automation Client API, interacts with it, clicking buttons “2″, “+”, “3″ and “=”.

Then UIA Explorer, a .NET- based application using UI Automation Client API, also interacts with gcalctool, if you pay attention you will notice that UIA Explorer also lists all other Gtk+ based applications, such as Nautilus and the GNOME Panel.

Finally Mozilla Firefox is executed (both Novell Moonlight with Accessibility Support and Novell Moonlight Accessibility Extensions plugins were installed prior running it) and an Atk-based Accessility Debugger, Accerciser, is launched to interact with the Moonlight application.

The source code used to test GCalctool is the following:

// gmcs -pkg:mono-uia gcalc.cs && mono gcalc.exe

using System;
using System.Linq;
using System.Windows.Automation;

namespace Mono.A11y.Examples {
	public class GCalc {
		public static void Main (string []args)
			AndCondition cond
				= new AndCondition (new PropertyCondition (AutomationElementIdentifiers.ControlTypeProperty,
				                    new PropertyCondition (AutomationElementIdentifiers.NameProperty, "Calculator"));
			var gcalc = AutomationElement.RootElement.FindFirst (TreeScope.Children, cond);
			cond = new AndCondition (new PropertyCondition (AutomationElementIdentifiers.ControlTypeProperty,
			                         new PropertyCondition (AutomationElementIdentifiers.NameProperty, "Add"));
			var addButton = gcalc.FindFirst (TreeScope.Descendants, cond);
			cond = new AndCondition (new PropertyCondition (AutomationElementIdentifiers.ControlTypeProperty,
			                         new PropertyCondition (AutomationElementIdentifiers.NameProperty, "Numeric 2"));
			var Button2 = gcalc.FindFirst (TreeScope.Descendants, cond);
			cond = new AndCondition (new PropertyCondition (AutomationElementIdentifiers.ControlTypeProperty,
			                         new PropertyCondition (AutomationElementIdentifiers.NameProperty, "Numeric 3"));
			var Button3 = gcalc.FindFirst (TreeScope.Descendants, cond);
			cond = new AndCondition (new PropertyCondition (AutomationElementIdentifiers.ControlTypeProperty,
			                         new PropertyCondition (AutomationElementIdentifiers.NameProperty, "Calculate result"));
			var calcButton = gcalc.FindFirst (TreeScope.Descendants, cond);
			cond = new AndCondition (new PropertyCondition (AutomationElementIdentifiers.ControlTypeProperty,
			                         new PropertyCondition (AutomationElementIdentifiers.NameProperty, "Calculate result"));
			var resultText = gcalc.FindFirst (TreeScope.Descendants, cond);

			// Clicking buttons
			InvokePattern addInvoke = (InvokePattern) addButton.GetCurrentPattern (InvokePattern.Pattern);
			InvokePattern Invoke2 = (InvokePattern) Button2.GetCurrentPattern (InvokePattern.Pattern);
			InvokePattern Invoke3 = (InvokePattern) Button3.GetCurrentPattern (InvokePattern.Pattern);
			InvokePattern calcInvoke = (InvokePattern) calcButton.GetCurrentPattern (InvokePattern.Pattern);
			System.Console.WriteLine ("Pressing \"2\"");
			System.Threading.Thread.Sleep (500);
			Invoke2.Invoke ();
			System.Console.WriteLine ("Pressing \"+\"");
			System.Threading.Thread.Sleep (500);
			addInvoke.Invoke ();
			System.Console.WriteLine ("Pressing \"3\"");
			System.Threading.Thread.Sleep (500);
			Invoke3.Invoke ();
			System.Console.WriteLine ("Pressing \"=\"");
			System.Threading.Thread.Sleep (500);
			calcInvoke.Invoke ();

Where do I get it?

If you installed Mono Accessibility 2.0 an option to upgrade should be available now, if not Mono Accessibility is available for a variety of Linux distributions, including:

Moonlight applications require the following extra steps:

Have fun and if you find any bug with this release, please file it. If you want to contribute or need specific assistance, please join our mailing list, or drop in #mono-a11y on

* Of course this depends on what APIs you are using. If you want to know how compatible is your application try MoMA.
** Notice however, even though Mono is muti platform and UI Automation is included in both Mono and Silverlight, in order to allow other ATs running on other platforms, different than Linux, you will require to implement an specific bridge to talk to you OS Accessibility Layer.

Shorten URL: url

1 Comment

RSS feed for comments on this post.

Sorry, the comment form is closed at this time.

This work is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 Unported License.
(c) 2004-2014 Mario Carrion | powered by WordPress with Barecity