ARchi VR Actions

Revision Info: Documentation for ARchi VR Version 2.2 - April 2020

Table of Contents

Up to ARchi VR Content Creation


Action

An action contains two parts: a list of new items to be inserted into the scene by the do : add task (see Task section below), and a sequence of tasks that specify the position or behavior of new or existing items. The result of a service call is an action with the following JSON data structure:

{ "items" : [ { "id" : "com.company.archi.meter", "vertices" : [ [ 0.0, 0.0, 0.0 ], [ 1.0, 0.0, 0.0 ] ], "type" : "Route", "subtype" : "Distance" } ], "tasks" : [ { "do" : "add", "id" : "com.company.archi.meter", "ahead" : "-0.5 0.0 -1.0" } ] }

Item

Item definitions are documented in ARchi VR Elements.

A good way to get a sample data structure of an item type is to create it within ARchi VR and then analyze the generated JSON code (e.g., sent by POST:SPACE).

Task

Tasks represent the position or behavior of items. do : add adds new items to the AR scene, but all other tasks manipulate existing items. The following tasks are currently supported by ARchi VR:

[ { "do" : "add", "id" : "_", "at" : "2.0 4.0 5.0" // absolute coordinates }, { "do" : "add", "id" : "_", "ahead" : "-1.0 0.0 -2.0" // relative to user position and orientation }, { "do" : "remove", "id" : "_" }, { "do" : "pulsate", "id" : "_" }, { "do" : "depulsate", "id" : "_" }, { "do" : "billboard", "id" : "_" }, { "do" : "highlight", "id" : "_" }, { "do" : "dehighlight", "id" : "_" }, { "do" : "hide", "id" : "_" }, { "do" : "unhide", "id" : "_" }, { "do" : "move", "id" : "_", "to" : "2.0 4.0 5.0" // absolute }, { "do" : "move", "id" : "_", "by" : "0.1 0.5 0.3" // relative }, { "do" : "turn", "id" : "_", "to" : "90.0" // absolute y rotation in degrees }, { "do" : "turn", "id" : "_", "by" : "-45.0" // relative y rotation in degrees }, { "do" : "translate", "id" : "_", "to" : "1.0 0.0 -1.0" }, { "do" : "translate", "id" : "_", "by" : "0.0 0.75 0.0" }, { "do" : "rotate", "id" : "_", "to" : "3.141 0.0 0.0" // absolute coordinates in radians }, { "do" : "rotate", "id" : "_", "by" : "0.0 -1.5705 0.0" // relative coordinates in radians }, { "do" : "classify", "id" : "_", "type" : "Interiour", // optional "subtype" : "Couch", // optional "name" : "Red Sofa" // optional }, { "do" : "save" // save the AR scene }, { "do" : "exit" // exit AR view without saving }, { "do" : "snapshot" // take screen snapshot }, { "do" : "say", "text" : "Good morning" }, { "do" : "play", "snd" : "1113" }, { "do" : "stream", "url" : "https://____.mp3", "volume" : "0.5" // NOTE: volume setting is currently not working }, { "do" : "pause" // pause stream }, { "do" : "service", "id" : "_" }, { "do" : "workflow", "id" : "_" }, { "do" : "execute", "op" : "function(...)" }, { "do" : "confirm", // get a YES/NO response as confirmation via an GUI popup window "dialog" : "Do you want to ...?", "then" : "function(...)", "else" : "function(...)" // optional "yes" : "Option 1" // optional, default "Yes" "no" : "Option 2" // optional, default "No" }, { "do" : "assign", // set a numeric variable "data" : "varname", // field name, key "to" : "_" // MUST be a string which will be converted to an integer or a float value }, { "do" : "set", // set a string variable "data" : "varname", // field name, key "to" : "_" // value as a string } ]

For all tasks, x-y-z parameters are interpreted according to the right-handed rule of the coordinate system. An optional on parameter defines a relative reference point. By default, it is "on" : "Floor", but can be set to "Ceiling", "Wall", "Item" (one that is ahead of user) or to an item id.

The "do" : "add" task adds a new item to the scene. The id corresponds to an item in the items list, and the position is specified with an x-y-z coordinate by at (places the item in absolute world space) or ahead (places the item relative to the user's position and orientation). The height represented by the y-coordinate is relative to the floor by default, but can also be relative to the ceiling height or the top of an item depending on the aforementioned on parameter.

For example the following json will place an item on the wall in front of the user at height 1. If no wall is ahead, then the item will be placed on the floor.

{ "do" : "add", "id" : "com.company.archi.meter", "ahead" : "0.0 1.0 0.0", "on" : "Wall" }

This task will place the flower item on the table item:

{ "do" : "add", "id" : "com.company.archi.flower", "at" : "-0.75 0.0 -0.40", "on" : "com.company.archi.table" }

The "do" : "move" and "do" : "turn" tasks manipulate the model elements and the corresponding 3D visualization nodes. The "do" : "turn" task turns in degrees clockwise.

The "do" : "translate" and "do" : "rotate" tasks only change the visual 3D nodes (and not the model item itself). The "do" : "rotate" task rotates in radians and counter-clockwise as euler angles.

The "do" : "say" task will use text-to-speech technology to read the given text aloud.

The "do" : "play" task needs a SystemSoundID number for the snd parameter. See the list of Apple iOS System Sound IDs for more information.

The "do" : "execute" task runs functions which are explained in the Interaction chapter below.

Sample Actions

Example 1 - Place a warning on the floor in front of the user:

{ "items" : [ { "id" : "net.metason.archi.warning", "vertices" : [ [ 0.0, 0.0, 0.0 ] ], "type" : "Spot", "subtype" : "Warning", "name" : "Attention" } ], "tasks" : [ { "do" : "add", "id" : "net.metason.archi.warning", "ahead" : "0.0 0.0 -1.0" } ] }

Example 2 - Place a 3D object in front of the user:

{ "items" : [ { "id" : "net.metason.archi.test2.asset1", "assetData" : "https://archi.metason.net/catalog/3D/A-Table.usdz", "assetMeta" : "model: A-Table;wxdxh: 2.35x0.88x0.71;scale: 0.01;", "type" : "Interiour", "subtype" : "Table" } ], "tasks" : [ { "do" : "add", "id" : "net.metason.archi.test2.asset1", "ahead" : "-0.5 0.0 -1.5" } ] }

Example 3 - Place a text panel at a fix position and turn it to 90 degrees:

{ "items" : [ { "id" : "com.company.archi.textpanel", "type" : "Spot", "subtype" : "Panel", "vertices" : [ [ 0.0, 0.0, 0.0 ] ], "assetData" : "<b>Hello</b><br>Welcome to ARchi VR.<br><small><i>Augmented Reality</i> at its best.</small>", "assetMeta" : "color:#DDCCFF; bgcolor:#333333DD; scale:2.0" } ], "tasks" : [ { "do" : "add", "id" : "com.company.archi.textpanel", "at" : "1.0 0.75 -2.0" }, { "do" : "turn", "id" : "com.company.archi.textpanel", "to" : "90.0", } ] }

See chapter Examples of Service Extensions for more sample code.

Temporal Task and Task Sequences

The execution of tasks can be temporally controlled with the in parameter which defines the delay in seconds. If the in parameter is not specified, the task will execute immediately.

{ "do" : "remove", "id" : "_", "in" : "2.5" }

Each task can be placed on the timeline so that time-controlled sequences of tasks can be defined.

Animation

The "do" : "animate" task creates a constant animation of the graphical node of an item (the model item itself stays unchanged).

{ "do" : "animate", "id" : "_", "key" : "position.y", "from" : "1.0", "to" : "1.5", "dur" : "1.5", "repeat" : "INFINITE", "reverse" : "true" }

The dur parameter specifies either the basic duration or interval of the animation in seconds.

The repeat parameter may be a fractional number of how often the animation interval is repeated. If set to "INFINITE", the animation will repeat forever.

The "do" : "stop" task removes an animation with the given key from the graphical node of an item.

{ "do" : "stop", "id" : "_", "key" : "position.y" }

An example of an animated action:

{ "items" : [ { "id" : "com.company.archi.box", "version" : 1, "type" : "Geometry", "subtype" : "Box", "assetMeta" : "color:#FF0000; wxdxh:0.4x0.4x0.7" } ], "tasks" : [ { "do" : "add", "id" : "com.company.archi.box", "ahead" : "-0.2 0.0 -1.0" }, { "do" : "animate", "id" : "com.company.archi.box", "key" : "eulerAngles.y", "from" : "0.0", "to" : "3.14", "dur" : "1.5", "repeat" : "INFINITE", "reverse" : "true" }, { "do" : "stop", "id" : "com.company.archi.box", "key" : "eulerAngles.y", "in" : "9.0" } ] }

Function Calls

Some of the tasks are also available as function calls which can be used for scripting. Functions can be sequenced by separating with a semicolon ';'.

The following functions are available for loading new content (as Actions) into the scene:

function('https://___', 'getJSON') // get a static Action function('https://___', 'postUser') // post User function('https://___', 'postSpace') // post Space function('https://___', 'postUserSpace') // post User & Space function('https://___', 'postCam') // post screen image function('https://___', 'postUserCam') // post User & screen

The following functions are available for manipulating the model using an item's id or for setting user interface (UI) elements:

function('id', 'delete') // remove item from scene function('id', 'turnto:', 90.0) // absolute y rotatation of item function('id', 'turnby:', -5.0) // relative y rotatation of item function('id', 'hightlight') // hightlight item function('id', 'dehightlight') // dehightlight item function('id', 'pulsate') // pulsate item function('id', 'depulsate') // depulsate item function('id', 'hide') // hide item or UI element function('id', 'unhide') // unhide item or UI element function('text', 'status') // set status text in UI function('text', 'warning') // set warning text in UI function('scene', 'save') // save AR session function('text', 'say') // talk via TTS function('1106', 'play') // play System Sound function('https://___.mp3', 'stream') // stream audio

If the id for delete ends with an asterix (*), such as in "function('net.metason.demo.*', 'delete')", then it will be interpreted as a wildcard and will delete all items with their id starting with the term before the asterix. A typical use case is that elements are removed with the delete function before an new action will load new items using a get/post function.

The id for UI elements in the AR view are

You may use wildcard asterix (*) notation for the id, so UI.* would select all UI elements for the hide or unhide task.

Interaction

The "content" : "ontap=___" declaration makes it possible to attach a click listener to 3D items. If the content parameter is set, the visual representation will pulsate to depict its interactive status. One ontap listener can execute multiple functions, separated with a semicolon (;). If an interactive item is tapped, the function(s) will be executed in order.

The following interactive triggers are examples of function calls for requesting actions via an URL with different POST contents:

"content" : "ontap=function('https://___', 'getJSON')" "content" : "ontap=function('https://___', 'postUser')" "content" : "ontap=function('https://___', 'postSpace')"

Interaction Icons

There are default icons available to create interactive buttons using image panels. The base URL to these icon images is https://service.metason.net/ar/extension/images/.

File Name Icon Usage
next.png next, start, forward, choose
back.png go backward
up.png up
down.png down
start.png start
stop.png stop, end service
docu.png show web page in pop-up window
msg.png show text message in small pop-up window
play.png stream audio
talk.png speech, say something (e.g., using text-to-speech or audio)
yes.png yes
no.png no, delete

Of course, you are free to use your own web server to provide custom icon images.

Dialog

The confirm: function opens a dialog window to get permission to run embedded function calls:

"content" : "ontap=function('Will you do this?', 'confirm:', 'function(\'___\', \'___\')'" // if confirmed then execute

Hint: Do NOT miss the colon in confirm: , otherwise no dialog appears. For embedded function calls, use escape quotes with a \'.

Data Variables

The following functions can be used to dynamically set data variables that are valid throughout an AR session:

"function('data.var1', 'assign:', 5.25)" // assign a numeric value (integer or float) to a variable "function('data.var2', 'set:', 'hello')" // set a string variable

Hint: Do NOT miss the colon in assign: and in set:, otherwise the variables will not be changed. Data variables will automatically be created when they do not already exist.

Data variables may be used in the precondition of dynamically loaded extensions, e.g., triggered by user-interaction with "content" : "ontap=function('https://___/__.json', 'getJSON')".

{ ... "precondition" : "data.var1 > 5.0", }, { ... "precondition" : "data.var2 == 'hello'", }

Examples of Service Extensions

Interactive Service Samples

The Welcome curation as well as the extensions listing contains "Service Samples" as an interactive tour through some examples demonstrating various items and tasks of service extensions.

   

Source Code of Service Samples

Study the source code of "Service Samples" listed below to learn how to build your own extension. It is referenced by the URL https://service.metason.net/ar/extension/metason/samples/.

The extension resource ext.json for "Service Samples" contains two app extensions:

  1. for the Service itself starting with start.json and
  2. for a Workflow that adds an attachment with a link to this documentation (docu.json ) if the session will be saved and contains sample elements.

The base URL to the JSON source code files and referenced content files is https://service.metason.net/ar/content/samples/.

The actions of "Service Samples" are listed below and cover key concepts of the Service extension for the ARchi VR App. Each action does call the next action by executing "content" : "ontap=function('https://___', 'getJSON')" when the user taps on the interactive "forward" button:


NOT YET IMPLEMENTED STUFF

Outlook to upcoming features in a future release of ARchi VR:

New Tasks

{ "dispatch" : "atstart", "do" : "enable", "system" : "Voice Guide" }, { "do" : "disable", "system" : "Voice Guide" }, { "do" : "enable", "system" : "Sonification" }, { "do" : "disable", "system" : "Sonification" }, { "do" : "enable", "system" : "Physics" }, { "do" : "disable", "system" : "Physics" }, { "do" : "unbillboard", "id" : "_" }

Conditional Task

Conditional tasks execute actions based on a are conditional trigger. Condtional triggers can be used for:

NOT YET IMPLEMENTED!

The condition is defined in the if expression. The task will only execute when the condition is evaluated as true. For if-then-else statements use two conditional tasks with complementary conditions.

{ "dispatch" : "onchange", "if" : "walls.@count == 1", "do" : "say", "text" : "Add next walls of room." }

The dispatch parameter defines the scope when the if expression of a condition task will be evaluated. Possible dispatch values are:

"dispatch" : "atstart", // at start of session before any capturing of space "dispatch" : "once", // only once within the task sequence, is default value when not defined "dispatch" : "onchange", // on each change of the space model "dispatch" : "always", // several times per seconds (~4 times per second) "dispatch" : "stated", // like "always", but action only is triggered when condition result is altered "dispatch" : "repeated", // as always with a given interval

The data elements accessible in the if expression are:

// local date and time of device as integer number time.year time.month // 1-12 time.day // 1-31 time.hour // 0-11 or 0-23 time.min // 0-59 time.sec // 0-59 // device info dev.type // Phone, Tablet, HMD dev.use // held (hand-held), worn (HMD) dev.screen.width dev.screen.height // device position & orientation held or worn by user user.pos.x // in meters user.pos.y user.pos.z user.rot.x // euler angles user.rot.y user.rot.z // space/room data walls // array of wall elements cutouts // array of cutouts (doors, windows) items // array of items docLinks // array of document links data // temporary variables

The following functions may be available in future releases for triggering actions:

"function('id', 'visible')" // 0 or 1 "function('id', 'proximity')" // distance in meters from camera to item with id "function('id', 'gazingAt')" // time in seconds gazing at item with id
{ "dispatch" : "stated", "if" : "function('id', 'proximity') < 1.2", "do" : "execute", "op" : "function('https://___', 'getJSON')" // run an action }

Dynamic GUI

A GUI alert for getting a YES/NO response as confirmation.

{ "dispatch" : "onchange", "if" : "items.@count == 20", "do" : "confirm", "dialog" : "Would you like to save the scene?", "then" : "function('scene', 'save')" }

NOT YET IMPLEMENTED!

Voice-based Dialog

A voice-based dialog for getting a YES/NO response via speech recognition.

{ "dispatch" : "onchange", "if" : "function('id', 'proximity') < 1.2", "do" : "ask", "question" : "Can I give you a hint?", "then" : "function('Tap on that pulsating item.', 'say')" }

NOT YET IMPLEMENTED!

Remote Analyzer

A customer-specific Analyzer is conditionally triggered after a new item is created by the user in order to classify it. An Analyzer may use machine learning algorithms (e.g., image analysis with computer vision or deep neural networks, geometric rules, decision trees, etc.) on a server to classify or even position/orient items in a scene.

NOT YET IMPLEMENTED!


Back to ARchi VR Content Creation


Copyright © 2020 Metason - All rights reserved.