ARchi VR Actions

Revision Info: Documentation for ARchi VR Version 2.3 - June 2020

Table of Contents

Up to ARchi VR Content Creation


Action

An action contains two parts: a list of new items to be inserted into the scene by the do : add task (see Task section below), and a sequence of tasks that specify the position or behavior of new or existing items. The result of a service call is an action with the following JSON data structure:

{ "items" : [ { "id" : "com.company.archi.meter", "vertices" : [ [ 0.0, 0.0, 0.0 ], [ 1.0, 0.0, 0.0 ] ], "type" : "Route", "subtype" : "Distance" } ], "tasks" : [ { "do" : "add", "id" : "com.company.archi.meter", "ahead" : "-0.5 0.0 -1.0" } ] }

Item

Item definitions are documented in ARchi VR Elements.

A good way to get a sample data structure of an item type is to create it within ARchi VR and then analyze the generated JSON code (e.g., sent by POST:SPACE).

Task

Tasks represent the position or behavior of items. do : add adds new items to the AR scene, but all other tasks manipulate existing items. The following tasks are currently supported by ARchi VR:

[ { "do" : "add", "id" : "_", "at" : "2.0 4.0 5.0" // absolute coordinates }, { "do" : "add", "id" : "_", "ahead" : "-1.0 0.0 -2.0" // relative to user position and orientation }, { "do" : "remove", "id" : "_" }, { "do" : "lock", "id" : "_" }, { "do" : "unlock", "id" : "_" }, { "do" : "pulsate", "id" : "_" }, { "do" : "depulsate", "id" : "_" }, { "do" : "billboard", "id" : "_" }, { "do" : "highlight", "id" : "_" }, { "do" : "dehighlight", "id" : "_" }, { "do" : "hide", "id" : "_" }, { "do" : "unhide", "id" : "_" }, { "do" : "move", "id" : "_", "to" : "2.0 4.0 5.0" // absolute }, { "do" : "move", "id" : "_", "by" : "0.1 0.5 0.3" // relative }, { "do" : "turn", "id" : "_", "to" : "90.0" // absolute y rotation in degrees }, { "do" : "turn", "id" : "_", "by" : "-45.0" // relative y rotation in degrees }, { "do" : "translate", "id" : "_", "to" : "1.0 0.0 -1.0" }, { "do" : "translate", "id" : "_", "by" : "0.0 0.75 0.0" }, { "do" : "rotate", "id" : "_", "to" : "3.141 0.0 0.0" // absolute coordinates in radians }, { "do" : "rotate", "id" : "_", "by" : "0.0 -1.5705 0.0" // relative coordinates in radians }, { "do" : "disable", "system" : "Sonification" }, { "do" : "unbillboard", "id" : "_" }, { "do" : "enable", "system" : "Voice Guide" }, { "do" : "disable", "system" : "Voice Guide" }, { "do" : "enable", "system" : "Sonification" }, { "do" : "skip", // skip wall capturing }, { "do" : "classify", "id" : "_", "type" : "Interiour", // optional "subtype" : "Couch", // optional "name" : "Red Sofa" // optional }, { "do" : "snapshot" // take screen snapshot }, { "do" : "say", "text" : "Good morning" }, { "do" : "play", "snd" : "1113" // system sound ID }, { "do" : "stream", "url" : "https://____.mp3", "volume" : "0.5" // NOTE: volume setting is currently not working }, { "do" : "pause" // pause current stream }, { "do" : "execute", "op" : "function(...)" }, { "do" : "service", "id" : "_" }, { "do" : "workflow", "id" : "_" }, { "do" : "fetch", "params" : "$0 = location.city; $1 = location.countryCode", "url" : "https://___/do?city=$0&country=$1", "map" : "data.var1 = result.key1; data.var2 = result.key2;" }, { "do" : "prompt", // show instruction in a GUI popup window "title" : "_", "text" : "_", "button" : "_", // optional, default is "Continue" "img" : "_URL_", // optional, image URL }, { "do" : "confirm", // get a YES/NO response as confirmation via an GUI popup window "dialog" : "Do you want to ...?", "then" : "function(...)", "else" : "function(...)", // optional "yes" : "Option 1", // optional, default "Yes" "no" : "Option 2" // optional, default "No" }, { "do" : "set", // set a string variable "data" : "_varname", // field name, key "to" : "_" // value as a string }, { "do" : "assign", // set a numeric variable "data" : "_varname", // field name, key "to" : "_" // MUST be a string which will be converted to an integer or a float value }, { "do" : "save" // save the AR scene }, { "do" : "avoid save" // disable save dialog }, { "do" : "exit" // exit AR view without saving } ]

For all tasks, x-y-z parameters are interpreted according to the right-handed rule of the coordinate system. An optional on parameter defines a relative reference point. By default, it is "on" : "Floor", but can be set to "Ceiling", "Wall", "Item" (one that is ahead of user) or to an item id.

The "do" : "add" task adds a new item to the scene. The id corresponds to an item in the items list, and the position is specified with an x-y-z coordinate by at (places the item in absolute world space) or ahead (places the item relative to the user's position and orientation). The height represented by the y-coordinate is relative to the floor by default, but can also be relative to the ceiling height or the top of an item depending on the aforementioned on parameter.

For example the following json will place an item on the wall in front of the user at height 1. If no wall is ahead, then the item will be placed on the floor.

{ "do" : "add", "id" : "com.company.archi.meter", "ahead" : "0.0 1.0 0.0", "on" : "Wall" }

This task will place the flower item on the table item:

{ "do" : "add", "id" : "com.company.archi.flower", "at" : "-0.75 0.0 -0.40", "on" : "com.company.archi.table" }

The "do" : "move" and "do" : "turn" tasks manipulate the model elements and the corresponding 3D visualization nodes. The "do" : "turn" task turns in degrees clockwise.

The "do" : "translate" and "do" : "rotate" tasks only change the visual 3D nodes (and not the model item itself). The "do" : "rotate" task rotates in radians and counter-clockwise as euler angles.

The "do" : "say" task will use text-to-speech technology to read the given text aloud.

The "do" : "play" task needs a SystemSoundID number for the snd parameter. See the list of Apple iOS System Sound IDs for more information.

The "do" : "execute" task runs functions which are explained in the Interaction chapter below.

Sample Actions

Example 1 - Place a warning on the floor in front of the user:

{ "items" : [ { "id" : "net.metason.archi.warning", "vertices" : [ [ 0.0, 0.0, 0.0 ] ], "type" : "Spot", "subtype" : "Warning", "name" : "Attention" } ], "tasks" : [ { "do" : "add", "id" : "net.metason.archi.warning", "ahead" : "0.0 0.0 -1.0" } ] }

Example 2 - Place a 3D object in front of the user:

{ "items" : [ { "id" : "net.metason.archi.test2.asset1", "assetData" : "https://archi.metason.net/catalog/3D/A-Table.usdz", "assetMeta" : "model: A-Table;wxdxh: 2.35x0.88x0.71;scale: 0.01;", "type" : "Interiour", "subtype" : "Table" } ], "tasks" : [ { "do" : "add", "id" : "net.metason.archi.test2.asset1", "ahead" : "-0.5 0.0 -1.5" } ] }

Example 3 - Place a text panel at a fix position and turn it to 90 degrees:

{ "items" : [ { "id" : "com.company.archi.textpanel", "type" : "Spot", "subtype" : "Panel", "vertices" : [ [ 0.0, 0.0, 0.0 ] ], "assetData" : "<b>Hello</b><br>Welcome to ARchi VR.<br><small><i>Augmented Reality</i> at its best.</small>", "assetMeta" : "color:#DDCCFF; bgcolor:#333333DD; scale:2.0" } ], "tasks" : [ { "do" : "add", "id" : "com.company.archi.textpanel", "at" : "1.0 0.75 -2.0" }, { "do" : "turn", "id" : "com.company.archi.textpanel", "to" : "90.0", } ] }

See chapter Examples of Service Extensions for more sample code.

Temporal Task and Task Sequences

The execution of tasks can be temporally controlled with the in parameter which defines the delay in seconds. If the in parameter is not specified, the task will execute immediately.

{ "do" : "remove", "id" : "_", "in" : "2.5" }

Each task can be placed on the timeline so that time-controlled sequences of tasks can be defined.

Animation

The "do" : "animate" task creates a constant animation of the graphical node of an item (the model item itself stays unchanged).

{ "do" : "animate", "id" : "_", "key" : "position.y", "from" : "1.0", "to" : "1.5", "dur" : "1.5", "repeat" : "INFINITE", "reverse" : "true" }

The dur parameter specifies either the basic duration or interval of the animation in seconds.

The repeat parameter may be a fractional number of how often the animation interval is repeated. If set to "INFINITE", the animation will repeat forever.

The "do" : "stop" task removes an animation with the given key from the graphical node of an item.

{ "do" : "stop", "id" : "_", "key" : "position.y" }

An example of an animated action:

{ "items" : [ { "id" : "com.company.archi.box", "version" : 1, "type" : "Geometry", "subtype" : "Box", "assetMeta" : "color:#FF0000; wxdxh:0.4x0.4x0.7" } ], "tasks" : [ { "do" : "add", "id" : "com.company.archi.box", "ahead" : "-0.2 0.0 -1.0" }, { "do" : "animate", "id" : "com.company.archi.box", "key" : "eulerAngles.y", "from" : "0.0", "to" : "3.14", "dur" : "1.5", "repeat" : "INFINITE", "reverse" : "true" }, { "do" : "stop", "id" : "com.company.archi.box", "key" : "eulerAngles.y", "in" : "9.0" } ] }

Function Calls

Some of the tasks are also available as function calls which can be used for scripting. Functions can be sequenced by separating with a semicolon ';'.

The following functions are available for loading new content (as Actions) into the scene:

function('https://___', 'getJSON') // get a static Action function('https://___', 'postUser') // post User function('https://___', 'postSpace') // post Space function('https://___', 'postUserSpace') // post User & Space function('https://___', 'postCam') // post screen image function('https://___', 'postUserCam') // post User & screen

The following functions are available for manipulating the model using an item's id or for setting user interface (UI) elements:

function('_id', 'delete') // remove item with id from scene function('_id', 'replace:', '_withID') // replace item with another item function('_id', 'turnto:', 90.0) // absolute y rotatation of item function('id', 'turnby:', -5.0) // relative y rotatation of item function('_id', 'hightlight') // hightlight item function('_id', 'dehightlight') // dehightlight item function('_id', 'lock') // lock item function('_id', 'unlock') // unlock item function('_id', 'pulsate') // pulsate item function('_id', 'depulsate') // depulsate item function('_id', 'hide') // hide item or UI element in AR view function('_id', 'unhide') // unhide item or UI element in AR view function('_text', 'status') // set status text in UI function('_text', 'warning') // set warning text in UI function('Walls', 'skip') // skip wall capturing function('camera', 'enable') // enable camera button function('camera', 'disable') // disable camera button function('help', 'enable') // enable help button function('help', 'disable') // disable help button function('save', 'enable') // enable save button function('save', 'disable') // disable save button function('undo', 'enable') // enable undo button function('undo', 'disable') // disable undo button function('catalog', 'enable') // enable catalog button function('catalog', 'disable') // disable catalog button function('scene', 'save') // save AR session function('save', 'avoid') // avoid save of scene function('_url', 'exit') // exit AR session without save; optional: open url afterwards function('text', 'say') // talk via TTS function('once', 'vibrate') // one second vibration function('twice', 'vibrate') // two short consecutive vibrations function('_sndID', 'play') // play System Sound, e.g., 1106 function('https://___.mp3', 'stream') // stream audio

If the id for delete ends with an asterix (*), such as in "function('net.metason.demo.*', 'delete')", then it will be interpreted as a wildcard and will delete all items with their id starting with the term before the asterix. A typical use case is that elements are removed with the delete function before an new action will load new items using a get/post function.

The id for UI elements in the AR view are

You may use wildcard asterix (*) notation for the id, so UI.* would select all UI elements in the AR view for the hide or unhide task.

Interaction

The "content" : "ontap=___" declaration makes it possible to attach a click listener to items. If the content parameter is set, the visual representation will pulsate to depict its interactive status. One ontap listener can execute multiple functions, separated with a semicolon (;). If an interactive item is tapped, the function(s) will be executed in order.

The following interactive triggers are examples of function calls for requesting actions via an URL with different POST contents:

"content" : "ontap=function('https://___', 'getJSON')" "content" : "ontap=function('https://___', 'postUser')" "content" : "ontap=function('https://___', 'postSpace')"

Interaction Icons

There are default icons available to create interactive buttons using image panels. The base URL to these icon images is https://service.metason.net/ar/extension/images/.

File Name Icon Usage
next.png next, start, forward, choose
back.png go backward
up.png up
down.png down
start.png start
stop.png stop, end service
info.png show info / instruction in pop-up window
fix.png fix, repair
docu.png show document / web page in pop-up window
msg.png show text message in small pop-up window
play.png stream audio
talk.png speech, say something (e.g., using text-to-speech or audio)
yes.png yes
no.png no, delete

Of course, you are free to provide your custom icon images using your own web server.

Dialogs

The prompt: function shows a window with a title and a text and can for example be used to present instructions:

"content" : "ontap=function('Title', 'prompt:', 'Message text')"

The confirm: function opens a dialog window to get a user's decision for executing activities (embedded function calls):

"content" : "ontap=function('Will you do this?', 'confirm:', 'function(`___`, `___`)'" // if confirmed then execute "content" : "ontap=function('Will you do this?', 'confirm::', 'function(`___`, `___`)', 'function(`___`, `___`)'" // if confirmed then execute first else execute second function(s)

Hint: Do NOT miss the colon in prompt: or in confirm:, otherwise no window appears. Do NOT miss the two colons in confirm:: for the if-then-else dialog, otherwise no dialog appears.

For function calls embedded in a function call, use back quotes ` (single left ticks) for their string parameters.

Data Variables

Data Assignement

The assign: and set: functions can be used to dynamically set data variables that are valid throughout an AR session:

"function('data.var0', 'assign:', 5.25)" // assign a numeric value (integer or float) to a variable "function('data.var0', 'increase:', 0.25)" // increase a numeric value (integer or float) "function('data.var1', 'decrease:', 1)" // decrease a numeric value (integer or float) "function('data.var2', 'set:', 'hello')" // set a string variable "function('data|triggers|*', 'clear')" // clear all data or tiggers or both

Hint: Do NOT miss the colon in assign: and in set:, otherwise the variables will not be changed. Data variables will automatically be created when they do not already exist.

You may use data variables to manage state in user interaction, e.g. as in "content" : "ontap=function('data.done', 'assign:', 1);function(...)"

Data variables may be used in the precondition of dynamically loaded extensions, e.g., triggered by user-interaction with "content" : "ontap=function('https://___', 'getJSON')".

{ ... "preCondition" : "data.done == 1", }, { ... "preCondition" : "data.var1 > 5.0", }, { ... "preCondition" : "data.var2 == 'hello'", }

Conditional Functions

"op" : "function('data.val == 5', 'ifthen:', 'function(`___`, `___`)'" // if predicate evaluates to true then execute function "op" : "function('data.val == 5', 'ifthenelse::', 'function(`___`, `___`)', 'function(`___`, `___`)'" // if predicate evaluates to true then execute first else execute second function(s)

For function calls embedded in a function call, use back quotes ` (single left ticks) for their string parameters.

Conditional Tasks

Conditional tasks only execute when their condition is true. Condtional triggers can be used for:

A condition is defined in the if expression. The task will only execute when the condition is evaluated as true. For if-then-else statements use two conditional tasks with complementary conditions.

{ "dispatch" : "stated", "if" : "walls.@count == 1", "do" : "say", "text" : "Add next walls of room." }

The data elements accessible in the if expression are:

// space/room data floor // floor element walls // array of wall elements cutouts // array of cutouts (doors, windows) items // array of items links // array of document links data // temporary variables // location location.address location.city location.state location.country location.countryCode location.postalCode location.longitude location.latitude // local date and time of device as integer number time.year time.month // 1-12 time.day // 1-31 time.hour // 0-11 or 0-23 time.min // 0-59 time.sec // 0-59 time.runtime // runtime of AR session in seconds (float) time.hms // string of hours:minutes.seconds formatted as HH:mm.ss // device info dev.type // Phone, Tablet, HMD dev.use // held (hand-held), worn (HMD) dev.screen.width dev.screen.height // device position & orientation held or worn by user user.pos.x // in meters user.pos.y user.pos.z user.rot.x // euler angles user.rot.y user.rot.z

Dispatch Mode of Tasks

The dispatch parameter defines the temporal scope when a task will be executed. Some do control how the if expression of a conditional task is evaluated. Possible dispatch values are:

"dispatch" : "once", // only once within the task sequence, is default value when not defined "dispatch" : "atstart", // at start of session before any capturing of space "dispatch" : "onchange", // on each change of the space model "dispatch" : "oncatch", // on detection of element (qrcode, barcode, image/marker, text, object, face, body) "dispatch" : "always", // several times per seconds (~4 times per second) "dispatch" : "repeated", // like "always", but only triggered "each" seconds "dispatch" : "stated", // like "always", but task only is triggered once when if-condition result is altered from false to true "dispatch" : "steady", // like "stated", but task only is triggered when condition result stays true "for" a certain time in seconds. Use "reset" (restart) after x seconds to set state back to false, otherwise only triggered once (default). "dispatch" : "altered", // like "always", but task always is triggered when if-condition result is altered (either from false to true or from true to false)

Some examples of dispatched tasks:

{ "dispatch" : "atstart", "do" : "disable", "system" : "Voice Guide" }, { "dispatch" : "repeated", "each" : "60", // seconds "do" : "say", "text" : "Another minute." }, { "if" : "walls.@count == 1", "dispatch" : "stated", "do" : "say", "text" : "Add next walls of room." }, { "if" : "function('id', 'proximity') < 1.2", "dispatch" : "stated", "do" : "execute", "op" : "function('https://___', 'getJSON')" // run an action }, { "if" : "function('com.company.archi.redbox', 'gazingAt') == 1", "dispatch" : "steady", "for" : "2.5", // seconds "do" : "remove", "id" : "com.company.archi.redbox" }, { "if": "function('net.metason.archi.redbox', 'gazingAt') == 1", "dispatch": "steady", "for" : "2.0", // seconds "reset" : "3.0", // reset/restart after seconds "do": "play", "snd": "1113" }, { "if" : "data.isON == 1", "dispatch" : "altered", "do" : "update", "id" : "net.metason.archi.lamp" }

Spatial Functions

The following functions are available to be used in conditions:

"function('id', 'visible')" // 0 or 1 if item with id is visible by user "function('id', 'proximity')" // distance in meters from camera to item with id "function('id', 'gazingAt')" // 0 or 1 if gazing at item with id "function('id', 'within')" // 0 or 1 if user is within 2D region of item with id "function('latitude, longitude', 'distance')" // distance in meters from current place

An example of a spatial condition for testing proximity, in this case unhiding an element when the user gets near to it:

{ "dispatch" : "stated", "if" : "function('_id', 'proximity') < 1.2", "do" : "execute", "op" : "function('_id_', 'unhide')" }

The latitude-longitude coordinates are in decimal degrees (e.g., '39.40338, 1.27403') or in degrees, minutes, and seconds using the backslash (\) escape character for the single and double quotes (e.g., 41°24'12.2"N 2°10'26.5"E as '41°24\'12.2\"N 2°10\'26.5\"E').

{ "dispatch" : "stated", "if" : "function('47.3769, 8.5417', 'distance') < 30.0", "do" : "execute", "op" : "function('https://___', 'getJSON')" // run an action }

Dynamic GUI

A GUI alert for getting a YES/NO response as confirmation.

{ "dispatch" : "onchange", "if" : "items.@count == 20", "do" : "confirm", "dialog" : "Would you like to save the scene?", "then" : "function('scene', 'save')", "else" : "function('once', 'vibrate')", // optional "yes" : "Yes", // optional confirmation text "no" : "No" // optional cancel text }

Examples of Service Extensions

Interactive Service Samples

The Welcome curation as well as the extensions listing contains "Service Samples" as an interactive tour through some examples demonstrating various items and tasks of service extensions.

   

Source Code of Service Samples

Study the source code of "Service Samples" listed below to learn how to build your own extension. It is referenced by the URL https://service.metason.net/ar/extension/metason/samples/.

The extension resource ext.json for "Service Samples" contains two app extensions:

  1. for the Service itself starting with start.json and
  2. for a Workflow that adds an attachment with a link to this documentation (docu.json ) if the session will be saved and contains sample elements.

The base URL to the JSON source code files and referenced content files is https://service.metason.net/ar/content/samples/.

The actions of "Service Samples" are listed below and cover key concepts of the Service extension for the ARchi VR App. Each action does call the next action by executing "content" : "ontap=function('https://___', 'getJSON')" when the user taps on the interactive "forward" button:


NOT YET IMPLEMENTED STUFF

Outlook to upcoming features in a future release of ARchi VR:

New Tasks

{ "do" : "enable", "system" : "Physics" }, { "do" : "disable", "system" : "Physics" }

Voice-based Dialog

A voice-based dialog for getting a YES/NO response via speech recognition.

{ "dispatch" : "onchange", "if" : "function('id', 'proximity') < 1.2", "do" : "ask", "question" : "Can I give you a hint?", "then" : "function('Tap on that pulsating item.', 'say')" }

NOT YET IMPLEMENTED!

Remote Analyzer

A custom Analyzer is conditionally triggered after a new item is created by the user in order to classify it. An Analyzer may use machine learning algorithms (e.g., image analysis with computer vision or deep neural networks, geometric rules, decision trees, etc.) on a server to classify or even position/orient items in a scene.

NOT YET IMPLEMENTED!


Back to ARchi VR Content Creation


Copyright © 2020 Metason - All rights reserved.