Web Scale collaboration and Crisis ManagementFrançois Charoy - @charoyInria/University of Lorraine

EQUIPE PROJETCoastCENTRE InriaNancy Grand Est 02/09/2015

Coast Team

• Inria/LORIA research team (15/20 people)

• University of Lorraine• Research topic : • Web Scale trustworthy

collaborative systems

• team.inria.fr/coast• @InriaLORIACoast

29/10/2015IscramMed 2015 Keynote – Large Scale Collaborative Systems and Crisis Management - 2

Inria NANCYGrand Est

29/10/2015IscramMed 2015 Keynote – Large Scale Collaborative Systems and Crisis Management - 3

Goals of the presentation

• Identify specific requirements for collaborative

systems in Crisis management

• Introduce some current results and research

challenges

• Introduce a new partner : the crowd

29/10/2015IscramMed 2015 Keynote – Large Scale Collaborative Systems and Crisis Management - 4

Disaster management phases

29/10/2015IscramMed 2015 Keynote – Large Scale Collaborative Systems and Crisis Management - 5

http://www.ifrc.org/en/what-we-do/disaster-management/about-disaster-management/

Emergency management organisation

29/10/2015IscramMed 2015 Keynote – Large Scale Collaborative Systems and Crisis Management - 6

• Multi-organisational :

–Security organisations,

–Health care organisation,

–Safety organisations,

–Government

organisations

• Centralised coordination

• Hierarchical

Collaboration partners

29/10/2015IscramMed 2015 Keynote – Large Scale Collaborative Systems and Crisis Management - 7

• Traditional actors

–Police,

–Firefighters,

–Healthcare, Hospitals,

–Civil Security,

–Military,

–Red Cross,

–Local Civil servants…

• Ad-Hoc actors– Industry worker,

– transportation workers

– Other GNOs

They collaborate with

29/10/2015IscramMed 2015 Keynote – Large Scale Collaborative Systems and Crisis Management - 8

• Email

• Phone

• Radio

• Other ICT Tools

When the scale increases

29/10/2015IscramMed 2015 Keynote – Large Scale Collaborative Systems and Crisis Management - 9

• Across borders

• Across organisations

• Across cultures

• Across interests

• Cluttered by

–Political and legal issues

–Limited resources

–Interoperability issues

It becomes messy

Variation in crisis

29/10/2015IscramMed 2015 Keynote – Large Scale Collaborative Systems and Crisis Management - 10

• Crisis management is an evolving matter• Recent crisis have obliged to revise the

preconceived view on crisis managementFrom a technical point of viewFrom a sociological point of view

Hurricane Katrina 2005

29/10/2015IscramMed 2015 Keynote – Large Scale Collaborative Systems and Crisis Management - 11

• Category 5 Hurricane • Caused breaches to flood protections structures• 80% of the city of New Orleans Submerged (1500 deaths)

Coordination issues

29/10/2015IscramMed 2015 Keynote – Large Scale Collaborative Systems and Crisis Management - 12

• Very wide area affected• Several political decision

levels• Lack of preparedness• Coordination issues

during the evacuation

“Some teams displayed their own initiative [..] Unfortunately, [..] rescuers were forced to leave people on highways [..] in continuing need of transportation, food, and water.”(White House Report / Hurricane Katrina)

Haiti Earthquake 2006

29/10/2015IscramMed 2015 Keynote – Large Scale Collaborative Systems and Crisis Management - 13

• 7.0 Magnitude earthquake – 3.5M people affected – 220.000 death (est.)

• Hundreds of responding organisations

Unexpected event at an unexpected scale

29/10/2015IscramMed 2015 Keynote – Large Scale Collaborative Systems and Crisis Management - 14

• First responder : people• Overwhelming answer

from the international community (hundreds of NGO’s)

• First important use of social network

• volunteer based crisis mapping

• http://www.digital-humanitarians.com/

Collaboration Challenges

29/10/2015IscramMed 2015 Keynote – Large Scale Collaborative Systems and Crisis Management - 15

• Social• Organisational• Technical

• Trust• Security• Safety• Interoperability• Culture• Scalability

Sharing and coordinating actions

Sharing informations

• Collaboration requires

–To be able to share information

•Effectively

•At a large scale

–To preserve privacy

–To trust the collaborators

–To control the data distribution

29/10/2015IscramMed 2015 Keynote – Large Scale Collaborative Systems and Crisis Management - 16

What do we share

DocumentsCalendarsMapsContactsPlans…

IscramMed 2015 Keynote – Large Scale Collaborative Systems and Crisis Management

Sharing with replication

29/10/2015 - 17

Reliability

•Possible to continue working when a replica crashes

•Better protection against corrupted data

Performance

•Scaling in numbers

•Scaling with the size of a geographical area

Challenge: how to maintain consistency between replicated data ?

. . .

Real-time Collaborative Editing Applications

29/10/2015IscramMed 2015 Keynote – Large Scale Collaborative Systems and Crisis Management

Collaborative activities:

–Report writing

–Transcription (lectures,

movie subtitles)

–Translation

–Note-taking during events

–Existing solutions:

–GoogleDocs, Office365,

Etherpad, Real-time editors for

Wikis (XWiki)

- 18

Limitations of RT Collaborative Technologies

Centralized control

Scalability issues (e.g. GoogleDocs 50 users sharing limit)

Privacy issues

29/10/2015IscramMed 2015 Keynote – Large Scale Collaborative Systems and Crisis Management - 19

IscramMed 2015 Keynote – Large Scale Collaborative Systems and Crisis Management

Conflicts management

29/10/2015 -

How to avoid them when the number of users grows

pouvais

pouvaitt

interfasses

interfacces

IscramMed 2015 Keynote – Large Scale Collaborative Systems and Crisis Management

Optimistic replication and Collaborative editing

29/10/2015 -

Testing scalability in GoogleDocs/MUTE

29/10/2015IscramMed 2015 Keynote – Large Scale Collaborative Systems and Crisis Management

Simulations

• Simulate users that type simultaneously on shared document

• Simulate different typing speeds

• Fast groups: 40 wpm

• Moderate groups: 35 wpm

• Slow groups: 23 wpm

• 1wpm=5cpm

• => 2-4 cps

Delays in GoogleDocs

29/10/2015IscramMed 2015 Keynote – Large Scale Collaborative Systems and Crisis Management

Typing speed: 2 chars/sec

- 24

Traditional centralized architecture

29/10/2015IscramMed 2015 Keynote – Large Scale Collaborative Systems and Crisis Management

Who control the server

- 25

Naive p2p Architecture

29/10/2015IscramMed 2015 Keynote – Large Scale Collaborative Systems and Crisis Management

- 26

Abstract/Configurable p2p architecture

29/10/2015IscramMed 2015 Keynote – Large Scale Collaborative Systems and Crisis Management

• Decide of the topology and the contracts

The Mute P2P editor

29/10/2015IscramMed 2015 Keynote – Large Scale Collaborative Systems and Crisis Management - 27

• Based on CRDT Algorithm (p2p)

http://mute-editorcrdt.rhcloud.com/

Delays in MUTE

29/10/2015IscramMed 2015 Keynote – Large Scale Collaborative Systems and Crisis Management

Max. no of users: 50 Typing frequency: 1-10 chars/s

Hybrid collaboration between organisations

29/10/2015IscramMed 2015 Keynote – Large Scale Collaborative Systems and Crisis Management - 29

• Centralised inside organisations/ p2p between organisations

Sharing Contract

Ad-Hoc collaboration on the field

29/10/2015IscramMed 2015 Keynote – Large Scale Collaborative Systems and Crisis Management - 30

• P2p collaboration between people regardless of organisations

MUTE P2P Editor

Web based ad-hoc collaboration

29/10/2015IscramMed 2015 Keynote – Large Scale Collaborative Systems and Crisis Management - 31

Reconnexion

1

2

Collaboration Man/Service

29/10/2015IscramMed 2015 Keynote – Large Scale Collaborative Systems and Crisis Management - 32

Storage Service (Drive/Github)

Document Service (translation,compilation)

Secure distributed collaboration

29/10/2015IscramMed 2015 Keynote – Large Scale Collaborative Systems and Crisis Management - 33

• Collaboration data can be encrypted and not shared on the cloud (being investigated).

• Provides guarantees to the participants

Future Work

Coordinating actions

29/10/2015IscramMed 2015 Keynote – Large Scale Collaborative Systems and Crisis Management - 34

• Coordination between organisation is a cornerstone of large scale collaboration.

• Important questionsWho governs the coordination ?Who owns the process ?Who design the process ?Who control the process ?

BPM and Crisis Management

29/10/2015IscramMed 2015 Keynote – Large Scale Collaborative Systems and Crisis Management - 35

• Business Process Management/Workflow Systems have been suggested for emergency management

• do they work for disaster management?

The answer is no/not really/probably not

29/10/2015IscramMed 2015 Keynote – Large Scale Collaborative Systems and Crisis Management - 36

• Outcome of a workshop with Firefighters

Four principles

29/10/2015IscramMed 2015 Keynote – Large Scale Collaborative Systems and Crisis Management - 37

• The model has to be dynamic• Design time = Execution time• Dependencies requires flexibility• No one should own all the control

Allen Temporal dependencies

Activity and dependency

29/10/2015IscramMed 2015 Keynote – Large Scale Collaborative Systems and Crisis Management

© SAP 2010 / Page 38

Search People

Plan

Execute

Finish Failed

Transport People

Plan

Execute

Finish Failed

ActivityActivity

Temporal Dependency

Idle

Activity and governance

29/10/2015IscramMed 2015 Keynote – Large Scale Collaborative Systems and Crisis Management - 39

Plan Exe-cute

Fail Finish Cancel

Idle ResponsibleAccountable, consulted

Accountable, responsible

Accountable, responsible

Accountable, Responsible,

Consulted

Responsible Role (User assignment to this role is part of the activity)

State of Manage-ment Lifecycle

Transition

Inter organisational issues

29/10/2015IscramMed 2015 Keynote – Large Scale Collaborative Systems and Crisis Management

© SAP 2008 / Page 40

“we are exchanging text information real-time through either cell or mobile units or satellite”[..]“obvious the more complex a situation gets, for instance where you may have many organizations working collectively in order to accomplish a goal, but at the same time working independently in terms of their own interests, where you get in those in situation like that, you know it get to the point, to be so complex that it would be overwhelming”

Fire Fighter, Hamilton, MT

Coordination problem awareness

29/10/2015IscramMed 2015 Keynote – Large Scale Collaborative Systems and Crisis Management - 41

Implementation on Google Wave

29/10/2015IscramMed 2015 Keynote – Large Scale Collaborative Systems and Crisis Management - 42

Experimentation

29/10/2015IscramMed 2015 Keynote – Large Scale Collaborative Systems and Crisis Management - 43

• How to validate the model ?

• Crisis are difficult to replicate

• Very few crisis management exercises

A new partner for crisis management: the citizen

29/10/2015IscramMed 2015 Keynote – Large Scale Collaborative Systems and Crisis Management - 44

• A challenge for traditional response organisation• They are the first responders• They have sophisticated communication tools• They have volunteers all over the world• It’s simple but it’s challenging

International and official recognition

29/10/2015IscramMed 2015 Keynote – Large Scale Collaborative Systems and Crisis Management - 45

• Since Haiti the role of social media and of digital volunteers is recognised

• Ushahidi• Digital Humanitarians• Visov (French organisation)• …

But…

Photos help NIWA map Christchurch flood

• Updating a flood map• Citizen + expert

coordination• Manual execution

http://tvnz.co.nz/national-news/photos-christchurch-floods-help-plan-future-video-5870548

29/10/2015IscramMed 2015 Keynote – Large Scale Collaborative Systems and Crisis Management - 46

Sandy Damage Assessment

• Crowdsourcing satellite pictures

• Results with short term validity

29/10/2015IscramMed 2015 Keynote – Large Scale Collaborative Systems and Crisis Management - 47

Sandy Map

29/10/2015IscramMed 2015 Keynote – Large Scale Collaborative Systems and Crisis Management - 48

How to make the best use of these resources

Wikipedia 1500 edit to the main page between the 28 and the 30th of october, not mentionning the talk or the area related pages

10 photos/s on instagram during sandy

Flickr – 3000 geolocalised photos/day

Twitter– 35000 geolocalised tweet/day

FEMA dedicated app

Unsollicited individual data

Dedicated app can receive notification and but used to ask for data

Collaborative data

Citizen Sourcing Data

CrowdExperts

Computers

INPUTS

Process

Large scale real time coordination

29/10/2015IscramMed 2015 Keynote – Large Scale Collaborative Systems and Crisis Management - 49

The Refugee crisis 2015

29/10/2015IscramMed 2015 Keynote – Large Scale Collaborative Systems and Crisis Management - 50

• A very tricky kind of crisis• Affect international relationships• Difficult agreement on the response

The #refugees response

29/10/2015IscramMed 2015 Keynote – Large Scale Collaborative Systems and Crisis Management - 51

Instagram stream – 23/9/2015 Twitter stream – 23/9/2015

More recent streams

29/10/2015IscramMed 2015 Keynote – Large Scale Collaborative Systems and Crisis Management - 52

Citizen initiative

29/10/2015IscramMed 2015 Keynote – Large Scale Collaborative Systems and Crisis Management - 53

• Citizen create Facebook pages to coordinate

• Different dimensions and scale

• Collect and distribute resources directly

IscramMed 2015 Keynote – Large Scale Collaborative Systems and Crisis Management

Crowdfunding

29/10/2015 - 54

• Citizen initiative• With or without NGO

support

Mapping information about aid initiatives

29/10/2015IscramMed 2015 Keynote – Large Scale Collaborative Systems and Crisis Management - 55

29/10/2015IscramMed 2015 Keynote – Large Scale Collaborative Systems and Crisis Management - 56

Mapping of events in Syria

29/10/2015IscramMed 2015 Keynote – Large Scale Collaborative Systems and Crisis Management - 57

Official vs citizen initiative

29/10/2015IscramMed 2015 Keynote – Large Scale Collaborative Systems and Crisis Management - 58

Airbnb for refugees

29/10/2015IscramMed 2015 Keynote – Large Scale Collaborative Systems and Crisis Management - 59

New challenges for Collaboration in crisis management

• Fast set up of large scale collaborative network

• Coordination at different scales

–People/Organisations

• Updating and sharing data at a large scale

–Trust/Privacy issues

• Involvement of people/The crowd

29/10/2015IscramMed 2015 Keynote – Large Scale Collaborative Systems and Crisis Management - 60

QUESTIONS ?

29/10/2015IscramMed 2015 Keynote – Large Scale Collaborative Systems and Crisis Management - 61

Credits

• Some slides have been prepared by

–Jorn Franke (Activity management)

–Gérald Oster (P2P sharing)

–Claudia Ignat (Collaborative Editing)

29/10/2015IscramMed 2015 Keynote – Large Scale Collaborative Systems and Crisis Management - 62