Semantics

Foreword

If you read some code related to Flutter, you will sometimes notice the use of Semantics or SemanticsConfiguration but the official documentation says very little on this interesting topic.

This article is an introduction to the topic and shows how important, interesting it might be for your application to take this into consideration.

In short - What is this about?

The official documentation says the following about the Semantics class:

A widget that annotates the widget tree with a description of the meaning of the widgetsUsed by accessibility tools, search engines, and other semantic analysis software to determine the meaning of the application.

I personally do not find this very self-explanatory. So I will use my own words:

In very short, the notion of Semantics is:

• totally optional (meaning that you could live without caring about it but not recommended),
• meant to be used in conjunction with Android TalkBack or iOS VoiceOver (e.g. mostly by visually impaired people)
• meant to be used by a Screen Reader that will describe the application without having to look at the screen.

By reading this, one can realize how this could be important if you target the application to also be usable by visually impaired people...

How is it implemented in Flutter?

When Flutter renders the Widgets tree, it also maintains a second tree, called Semantics Tree which is used by the Mobile Device assistive technology (Android TalkBack or iOS VoiceOver).

Each node of this Semantics tree is a SemanticsNode which might correspond to one or to a group of Widgets.

Each SemanticsNode is linked to a SemanticsConfiguration, a series of properties that tell the Mobile Device assistive technology how to:

• describe the node
• behave with the node

SemanticsConfiguration

Describes the semantic information associated with the owning SemanticsNode. Here follows some of the properties (for an exhaustive list, please refer to the official documentation).

Implicit Flutter Widgets with Semantics

Most of the Flutter Widgets are implicitly defined as Semantics since they all might be directly or indirectly used by the Screen Reader engine.

To illustrate this, here is an extract of the Flutter source code related to a Button:

class _RawMaterialButtonState extends State<RawMaterialButton> {

  ...

  @override
  Widget build(BuildContext context) {

    ...

    return Semantics(
      container: true,
      button: true,
      enabled: widget.enabled,
      child: ConstrainedBox(
        constraints: widget.constraints,
        child: Material(
          ...
        ),
      ),
    );
  }
}

How to define a Semantics?

Sometimes it might be interesting to define a part of the screen so that it could be described by the Mobile Device assistive technology.

In that case, simply use one of the following Widgets as a container of your sub-widget(s):

• Semanticswhen you want to describe only 1 particular Widget

• MergeSemanticswhen you want to describe a group of Widgets. In this case, the different Semantics which will be defined in the sub-tree of this node, will be merged into one single Semantics. This could be very useful to regroup semantics, however, in case of conflicting semantics, the result may be nonsensical.

Single Semantics

The class to be used to define a Semantics is Semantics. This class has 2 constructors: a verbose one or a concise one.

Here follows the 2 ways of defining a Semantics, explanation follows:

@override
Widget build(BuildContext context){
  bool toBeMergedWithAncestors = false;
  bool allowDescendantsToAddSemantics = false;

  return Semantics(
    container: toBeMergedWithAncestors,
    explicitChildNodes: allowDescendantsToAddSemantics,
    ...(list of all properties)...

    child: ...
  );
}

@override
Widget build(BuildContext context){
  SemanticsProperties properties = SemanticsProperties(...);
  bool isContainer = toBeMergedWithAncestors;
  bool explicitChildNodes = allowDescendantsToAddSemantics;

  return Semantics.fromProperties(
    container: isContainer,
    explicitChildNodes: explicitChildNodes,
    properties: properties,
    child: ...
  );
}

How to not have a Semantics?

Sometimes, there might be cases where you would not need any Semantics at all. This might be the case for parts of the screen which are only decorative, not important to the user.

In this case, you need to use the ExcludeSemantics class to exclude the Semantics of all this Widget's descendants. Its syntax is the following:

@override
Widget build(BuildContext context){
  bool alsoExcludeThisWidget = true;

  return ExcludeSemantics(
    excluding: alsoExcludeThisWidget,
    child: ...
  );
}

The excluding property (default: true), tells the system whether you also want this Widget to be excluded from the Semantics tree as well.

How to regroup Widgets into one single Semantics?

Under some circumstances, you might also want to regroup all the Semantics of a set of Widgets.

A basic example of such case might be a visual block made up of a Label and a Checkbox, each one defining its own Semantics. It would be preferable that if the user presses the block, the Mobile Device assistive technology would give assistance related to the group rather than to each Widget of the group.

In this case, you should use the MergeSemantics class.

WARNING

Be very careful when you want to merge the Semantics since if you have any conflicting Semantics, this might result in becoming nonsensical for the user. For example, if you have a block made up of several checkboxes, each of them having different statuses (checked and not checked), the resulting Semantics status will be checked, misleading the user.

How to debug the Semantics?

Finally, if you want to debug the Semantics of your application, you may set the showSemanticsDebugger property of your MaterialApp to true. This will force Flutter to generate an overlay to visualize the Semantics tree.

void main(){
  runApp(MyApp());
}

class MyApp extends StatefulWidget {
  @override
  _MyAppState createState() => _MyAppState();
}

class _MyAppState extends State<MyApp> {
  // This widget is the root of your application.
  @override
  Widget build(BuildContext context) {
    return MaterialApp(
      title: Text('My Semantics Test Application'),
      showSemanticsDebugger: true,
      theme: ThemeData(
        primarySwatch: Colors.blue,
      ),
      home: FirstScreen(),
    );
  }
}

Conclusions

As the official documentation is still not very verbose on this topic, I simply wanted to share my understanding with you.

I hope that this introduction highlighted the fact that it is important to consider the Semantics if you want to release an application one day as, Mobile users might turn on the Mobile Device assistive technology of their phone and use your application. If your application is not ready for this technology, there might be risks that it could not be used.

Stay tuned for next articles and happy coding.