USCoast

Presentation

Main.Presentation History

Hide minor edits - Show changes to output

Changed lines 3-7 from:
The project focuses on two substantive areas - real-time collaborative editing and trust-based collaboration-, with one overarching methodological contribution.

!!! Real-time collaborative editing
Real-time collaborative editing including an insightful understanding of real-time requirements for collaborative editing, grounded in a theory for the effect of real-time constraints in collaborative work. Current related work is fundamentally flawed, based on tasks with varied time constants, idiosyncratic task coupling and uncontrolled compensatory strategies. The project focusses as well on awareness management for the coordination of work in the presence of conflict and disruption.
to:
The project focuses on two substantive areas - collaborative editing and trust-based collaboration-, with one overarching methodological contribution.

!!! Collaborative editing
Collaborative editing including an insightful understanding of real-time requirements for collaborative editing, grounded in a theory for the effect of real-time constraints in collaborative work. Current related work is fundamentally flawed, based on tasks with varied time constants, idiosyncratic task coupling and uncontrolled compensatory strategies. The project focusses as well on non real-time or asynchronous collaborative editing and awareness management for the coordination of work in the presence of conflict and disruption.
Added lines 39-42:
!!!! Measurement of delays in real-time collaborative editing systems

By means of simulations we measured delays in popular real-time collaborative editing systems such as GoogleDocs and Etherpad in terms of the number of users that edit a shared document and their typing frequency. We simulated a variable number of users that contribute with a variable frequency to a shared document edited in collaboration by using GoogleDocs. We varied the number of users that collaborate on the document from 1 to 50 and the typing speed from 1 character/s to 10 characters/s. We measured the delay between the time when a modification is done by a simulated user and the time this modification is visible to the other simulated users. We measured that the delay is incrementing with the number of users and with the typing frequency. GoogleDocs does rarely support more than 30 clients that have an average typing speed. For 1 to 30 clients and for speeds of 1-10 characters/s, delays are ranging between 0 and 25 s. In Etherpad users are disconnected if the number of concurrent users is higher than 10. Results of this study were published at Internet of People Workshop 2016 organised in conjunction with Networking conference.

Changed lines 44-46 from:
One of the important goals of this associate team is to develop an experimental design for testing SCORE's trust-based collaboration model for a large community of users. To ground our findings in existing social science theory, we examined the game theory literature that spans cognitive science, psychology and economics. Some of the questions we worked on were: is there a game theory model that could reflect document sharing in collaborative editing? is there a game theory model that deals with user reputation? what task scenario can be proposed for experimental studies? Our paradigm builds on game theory methods in several ways. First, we adapted a well-established trust game to a repetitive setting that best suits to trust-based collaboration. Second, we added user attributes for trust, as required in the SCORE model, in order to determine their effect on the decision to collaborate. Trust values are updated based on the satisfaction level for the exchanges during the game.
to:
One of the important goals of this associate team is to develop an experimental design for testing SCORE's trust-based collaboration model for a large community of users. To ground our findings in existing social science theory, we examined the game theory literature that spans cognitive science, psychology and economics. Some of the questions we worked on were: is there a game theory model that could reflect document sharing in collaborative editing? is there a game theory model that deals with user reputation? what task scenario can be proposed for experimental studies? Our paradigm builds on game theory methods in several ways. First, we adapted a well-established trust game to a repetitive setting that best suits to trust-based collaboration. Second, we added user attributes for trust, as required in the SCORE model, in order to determine their effect on the decision to collaborate. Trust values are updated based on the satisfaction level for the exchanges during the game.

We designed a trust metric that reflects user behaviours during the interactions in trust game. A current trust value is computed based on the current iteration of the game. This current trust value is aggregated with computed trust values from previous iterations of the game involving the same partners. Aggregation of trust values during various iterations of the game reflects variations in user behaviour, i.e. our metric is robust against fluctuating user behavior. We validated our trust metric by using an empirical approach against data sets collected from several trust game experiments. We showed that our model is consistent with rating opinions of users, and our model can provide higher accuracy on predicting users’ behavior compared with other naive models. This result was published at TrustCom 2016
.
Changed line 40 from:
One of the important goals of this associate team is to develop an experimental design for testing SCORE's trust-based collaboration model for a large community of users. To ground our findings in existing social science theory, we examined the game theory literature that spans cognitive science, psychology and economics. Some of the questions we worked on were: is there a game theory model that could reflect document sharing in collaborative editing? is there a game theory model that deals with user reputation? what task scenario can be proposed for experimental studies? Our paradigm builds on game theory methods in several ways. First, we adapted a well-established trust game to a repetitive setting that best suits to trust-based collaboration. Second, we accommodated the need for a large community of users by involving simulated users as well as human participants in the experiment. Third, we added user attributes for trust, as required in the SCORE model, in order to determine their effect on the decision to collaborate. Trust values are updated based on the satisfaction level for the exchanges during the game.
to:
One of the important goals of this associate team is to develop an experimental design for testing SCORE's trust-based collaboration model for a large community of users. To ground our findings in existing social science theory, we examined the game theory literature that spans cognitive science, psychology and economics. Some of the questions we worked on were: is there a game theory model that could reflect document sharing in collaborative editing? is there a game theory model that deals with user reputation? what task scenario can be proposed for experimental studies? Our paradigm builds on game theory methods in several ways. First, we adapted a well-established trust game to a repetitive setting that best suits to trust-based collaboration. Second, we added user attributes for trust, as required in the SCORE model, in order to determine their effect on the decision to collaborate. Trust values are updated based on the satisfaction level for the exchanges during the game.
Added line 28:
Deleted line 27:
Changed lines 30-35 from:
Number of words in the text base
the keywords as a measure of document quality. Keywords is computed as the number of main keywords present in the final version of the document provided by each group of users. We examined the number of keywords divided by the number of words
redundancy is another measure of document quality. Redundancy is computed as the sum of redundancies of each section in the document. Redundancy of a section was measured by analysing the recorded videos of the collaborative editing session. Redundancy of a section represents the maximum number of occurrences in that section of any topic present in the audio.
Error Rate is another measure of document quality. Error rate is computed using Reverso tool. Reverso checks misspellings and grammar of a text in any language. We examined the number of errors divided by the number of words.
Chat Behavior was studied for measuring coordination. We examined the number of words, accord language, and definite determiners.
Survey responses. For instance we examined the experience of using collaborative editing of users. We divided the groups into high experienced and low experienced.
to:
* Number of words in the text base
* the keywords as a measure of document quality. Keywords is computed as the number of main keywords present in the final version of the document provided by each group of users. We examined the number of keywords divided by the number of words
* redundancy is another measure of document quality. Redundancy is computed as the sum of redundancies of each section in the document. Redundancy of a section was measured by analysing the recorded videos of the collaborative editing session. Redundancy of a section represents the maximum number of occurrences in that section of any topic present in the audio.
* Error Rate is another measure of document quality. Error rate is computed using Reverso tool. Reverso checks misspellings and grammar of a text in any language. We examined the number of errors divided by the number of words.
* Chat Behavior was studied for measuring coordination. We examined the number of words, accord language, and definite determiners.
* Survey responses. For instance we examined the experience of using collaborative editing of users. We divided the groups into high experienced and low experienced.
Changed lines 24-36 from:
For the sorting task we used measures of sorting accuracy based on the insertion sort algorithm, average time per entry, chat behavior, collisions between users, and crucially, task strategy. We note in the followings some key findings.

* What is the effect of delay on task process and outcome?
The general effect of delay is to slow down performance, which in this case hinders task outcome. Second
, the effect of a system property such as delay can be small relative to strategies. Third, the effect of delay need not be linear.  Complaint about delay was a low frequency event at the level of individual participants.  This makes sense because participants would have had to experience the direct consequence of delay in order to detect it.  Collisions are one such consequence, which depending upon strategy, participants did not always recognize and repair. Nevertheless, at the group level, most groups had at least one participant complain about delay at the highest levels.  

* What is the effect of strategy on task process and outcome?
The relationship between speed and insertion sort scores, is not in general a speed accuracy tradeoff.  Second, a strategy effect appears in the outcome measure, response time,  and interface ratings. Groups that sorted continuously (relative
to those who postponed sorting) liked the interface better than sort at the end groups, perhaps reflecting satisfaction with the higher level of task success. 

* How does strategy interact with delay to affect task process and outcome?
The effect of delay depends on strategy. However, somewhat counterintuitively, and unlike previous research, the overall superior strategy does not overcome the effect
of delay.  In fact, the insertion sort score declined with delay for continuous sort, but did not for sort-at-the-end.  We suspect that continuous sort entails more coupling, because years must be in place prior to positioning, and because text position is changing frequently throughout the entire task as sorting proceeds.  Even continuous-sort participants in the 0  and 4 second delay conditions complained about the ``jumping'' line positions.  To manage the coupling in continuous sort, we see participants slow down with delay. However, the negative slope on the insertion sort metric for continuous sort relative to the flat slope for sort-at-the-end suggests that the continuous sort strategy is only adaptive within a range of delay. Untested levels of delay could actually result in worse performance for continuous sorting than a sort-at-the-end strategy.  Furthermore, the chat metric suggests that sort-at-the-end requires more local coordination as delay increases. Thus the coordination established by formal agreement at the outset in continuous-sort appears to favor efficient communication over the ability to respond to local perturbations.  On the other hand, sort-at-the-end appears to favor the ability to respond to local perturbations at the expense of efficient communication.

!!!! Analysis of the interview note taking task
This task is under current work. The questions we target are: what is user acceptance
of delay in the note taking task? do users adapt to the delay in this type of task ? is performance of the group reduced in the presence of delay? are user conflicts more frequent in the presence of delay?
to:
For the sorting task we used measures of sorting accuracy based on the insertion sort algorithm, average time per entry, chat behavior, collisions between users, and crucially, task strategy (tightly coupled or loosely coupled task decomposition of the task).  We found out that delay slows down participants which decrements the outcome metric of sorting accuracy. Tightly coupled task decomposition enhances outcome at minimal delay, but participants slow down with higher delays. A loosely coupled task decomposition at the beginning leaves a poorly coordinated tightly coupled sorting at the end, requiring more coordination as delay increases. A paper describing the results was published in CDVE 2014.

!!!! Note taking task
The precise research questions for this study were how does
delay influence the quality of the final document in terms of the number of grammar errors present in the document, the amount of redundancy and the number of keywords present in the final document with respect to the transcript of the audio. We also wanted to answer to the question whether users try to adopt compensatory strategies to overcome delay by means of coordination over chat that we quantified according to the use of accord language and definite determiners. And finally how do delay, experience and compensatory collaboration effort interact to affect task performance.

As dependent measures we analysed:
• Number of words in
the text base
• the keywords as a measure of document quality. Keywords is computed as the number of main keywords present in the final version of the document provided by each group of users. We examined the number of keywords divided by the number of words
• redundancy is another measure
of document quality. Redundancy is computed as the sum of redundancies of each section in the document. Redundancy of a section was measured by analysing the recorded videos of the collaborative editing session. Redundancy of a section represents the maximum number of occurrences in that section of any topic present in the audio.
• Error Rate is another measure of document quality. Error rate is computed using Reverso tool. Reverso checks misspellings and grammar of a text
in any language. We examined the number of errors divided by the number of words.
• Chat Behavior was studied
for measuring coordination. We examined the number of words, accord language, and definite determiners.
• Survey responses. For instance we examined the experience
of using collaborative editing of users. We divided the groups into high experienced and low experienced.

We found that
the error rate is higher for groups that experienced a higher level of delay. Redundancy is higher for groups in higher delay condition. We also found out that as the delay increases the keywords depicted by users decreases. We separated the groups into high experienced and low experienced according to the data in the questionnaire. For high experienced groups redundancy increases with the delay, but for low experienced groups we could not see the same tendancy. We also measured chat behavior by means of number of accord words and definite determiners which together provides a common ground knowledge which we considered as a measure of coordination. We have seen that low experienced groups used more coordination to manage redundancy. High experienced groups did not adjust their collaboration effort to manage redundancy. Results of this task were published at ECSCW 2015.
Changed lines 9-10 from:
Trust-based collaboration including validation of proposed light security mechanisms for decentralised collaboration, based on posted measures of voluntary compliance with data sharing restrictions.
to:
Trust-based collaboration where users master and control their data by deciding with whom they share their data without relying on a central authority. We investigate new trust-based access control mechanisms where access is given based on user trust values that are dynamic and vary during a collaboration. These mechanisms are scalable and usable. The main questions we address are how to compute the trust values such that they correctly reflect collaboration experiences between users and that they are accepted by users.
Changed line 27 from:
The general effect of delay is to slow down performance, which in this case hinders task outcome. Second, we suggest that the effect delay is a function of the task time constant. In our case, with a task time constant on the order of 3.5 seconds, the effective manipulations are much larger than the 700 msec characteristic of motor control tasks, with problems becoming apparent at 8 and 10 seconds.  Third, the effect of a system property such as delay can be small relative to strategies. Fourth, the effect of delay need not be linear.  Complaint about delay was a low frequency event at the level of individual participants.  This makes sense because participants would have had to experience the direct consequence of delay in order to detect it.  Collisions are one such consequence, which depending upon strategy, participants did not always recognize and repair. Nevertheless, at the group level, most groups had at least one participant complain about delay at the highest levels. 
to:
The general effect of delay is to slow down performance, which in this case hinders task outcome. Second, the effect of a system property such as delay can be small relative to strategies. Third, the effect of delay need not be linear.  Complaint about delay was a low frequency event at the level of individual participants.  This makes sense because participants would have had to experience the direct consequence of delay in order to detect it.  Collisions are one such consequence, which depending upon strategy, participants did not always recognize and repair. Nevertheless, at the group level, most groups had at least one participant complain about delay at the highest levels. 
Changed line 5 from:
!!!Real-time collaborative editing
to:
!!! Real-time collaborative editing
Changed line 35 from:
!!! Analysis of the interview note taking task
to:
!!!! Analysis of the interview note taking task
Changed line 38 from:
!! Trust-based collaboration model
to:
!!! Trust-based collaboration model
Changed line 5 from:
!! Real-time collaborative editing
to:
!!!Real-time collaborative editing
Changed line 8 from:
!! Trust-based collaboration
to:
!!! Trust-based collaboration
Changed line 11 from:
!! Validation
to:
!!! Validation
Changed lines 14-16 from:
! Progress and ongoing work
!! Real-time collaborative editing
to:
!! Progress and ongoing work
!!! Real-time collaborative editing
Changed line 23 from:
!!! Sorting task
to:
!!!! Sorting task
Changed line 26 from:
!!!! What is the effect of delay on task process and outcome?
to:
* What is the effect of delay on task process and outcome?
Changed line 29 from:
!!!! What is the effect of strategy on task process and outcome?
to:
* What is the effect of strategy on task process and outcome?
Changed line 32 from:
!!!! How does strategy interact with delay to affect task process and outcome?
to:
* How does strategy interact with delay to affect task process and outcome?
Changed line 5 from:
!!! Real-time collaborative editing
to:
!! Real-time collaborative editing
Changed line 8 from:
!!! Trust-based collaboration
to:
!! Trust-based collaboration
Changed line 11 from:
!!! Validation
to:
!! Validation
Changed line 14 from:
!! Progress and ongoing work
to:
! Progress and ongoing work
Changed lines 15-16 from:
to:
!! Real-time collaborative editing
Changed line 38 from:
!!! Trust-based collaboration model
to:
!! Trust-based collaboration model
Changed lines 22-24 from:
Analysis thus far has focused on the sorting task, using measures of sorting accuracy based on the insertion sort algorithm, average time per entry, chat behavior, collisions between users, and crucially, task strategy. We note in the followings some key findings.

!!! What is the effect of delay on task process and outcome?
to:
!!! Sorting task
For the sorting task we used measures of sorting accuracy based on the insertion sort
algorithm, average time per entry, chat behavior, collisions between users, and crucially, task strategy. We note in the followings some key findings.

!!!! What is the effect of delay on task process and outcome?
Changed line 28 from:
!!! What is the effect of strategy on task process and outcome?
to:
!!!! What is the effect of strategy on task process and outcome?
Changed lines 31-32 from:
!!! How does strategy interact with delay to affect task process and outcome?
The effect of delay depends on strategy. However, somewhat counterintuitively, and unlike previous research, the overall superior strategy does not overcome the effect of delay.  In fact, the insertion sort score declined with delay for continuous sort, but did not for sort-at-the-end.  We suspect that continuous sort entails more coupling, because years must be in place prior to positioning, and because text position is changing frequently throughout the entire task as sorting proceeds.  Even continuous-sort participants in the 0  and 4 second delay conditions complained about the ``jumping'' line positions.  To manage the coupling in continuous sort, we see participants slow down with delay. However, the negative slope on the insertion sort metric for continuous sort relative to the flat slope for sort-at-the-end suggests that the continuous sort strategy is only adaptive within a range of delay. Untested levels of delay could actually result in worse performance for continuous sorting than a sort-at-the-end strategy.  Furthermore, the chat metric suggests that sort-at-the-end requires more local coordination as delay increases. Thus the coordination established by formal agreement at the outset in continuous-sort appears to favor efficient communication over the ability to respond to local perturbations.  On the other hand, sort-at-the-end appears to favor the ability to respond to local perturbations at the expense of efficient communication.
to:
!!!! How does strategy interact with delay to affect task process and outcome?
The effect of delay depends on strategy. However, somewhat counterintuitively, and unlike previous research, the overall superior strategy does not overcome the effect of delay.  In fact, the insertion sort score declined with delay for continuous sort, but did not for sort-at-the-end.  We suspect that continuous sort entails more coupling, because years must be in place prior to positioning, and because text position is changing frequently throughout the entire task as sorting proceeds.  Even continuous-sort participants in the 0  and 4 second delay conditions complained about the ``jumping'' line positions.  To manage the coupling in continuous sort, we see participants slow down with delay. However, the negative slope on the insertion sort metric for continuous sort relative to the flat slope for sort-at-the-end suggests that the continuous sort strategy is only adaptive within a range of delay. Untested levels of delay could actually result in worse performance for continuous sorting than a sort-at-the-end strategy.  Furthermore, the chat metric suggests that sort-at-the-end requires more local coordination as delay increases. Thus the coordination established by formal agreement at the outset in continuous-sort appears to favor efficient communication over the ability to respond to local perturbations.  On the other hand, sort-at-the-end appears to favor the ability to respond to local perturbations at the expense of efficient communication.

!!! Analysis of the interview note taking task
This task is under current work. The questions we target are: what is user acceptance of delay in the note taking task? do users adapt to the delay in this type of task ? is performance of the group reduced in the presence of delay? are user conflicts more frequent in the presence of delay?

!!! Trust-based collaboration model
One of the important goals of this associate team is to develop an experimental design for testing SCORE's trust-based collaboration model for a large community of users. To ground our findings in existing social science theory, we examined the game theory literature that spans cognitive science, psychology and economics. Some of the questions we worked on were: is there a game theory model that could reflect document sharing in collaborative editing? is there a game theory model that deals with user reputation? what task scenario can be proposed for experimental studies? Our paradigm builds on game theory methods in several ways. First, we adapted a well-established trust game to a repetitive setting that best suits to trust-based collaboration. Second, we accommodated the need for a large community of users by involving simulated users as well as human participants in the experiment. Third, we added user attributes for trust, as required in the SCORE model, in order to determine their effect on the decision to collaborate. Trust values are updated based on the satisfaction level for the exchanges during the game.
Changed line 19 from:
* A note taking task, in which participants listed to a 10 minute interview on the topic of cloud computing, and provided an integrated set of notes on the interview
to:
* A note taking task, in which participants listened to a 10 minute interview on the topic of cloud computing, and provided an integrated set of notes on the interview
Changed line 22 from:
Analysis thus far has focused on the sorting task, using measures of sorting accuracy based on the insertion sort algorithm, average time per entry, chat behavior, collisions between users, and crucially, task strategy. We note here some key findings:
to:
Analysis thus far has focused on the sorting task, using measures of sorting accuracy based on the insertion sort algorithm, average time per entry, chat behavior, collisions between users, and crucially, task strategy. We note in the followings some key findings.
Changed lines 4-6 from:
*Real-time collaborative editing including an insightful understanding of real-time requirements for collaborative editing, grounded in a theory for the effect of real-time constraints in collaborative work. Current related work is fundamentally flawed, based on tasks with varied time constants, idiosyncratic task coupling and uncontrolled compensatory strategies. The project focusses as well on awareness management for the coordination of work in the presence of conflict and disruption.
*''Trust-based collaboration'' including validation of proposed light security mechanisms for decentralised collaboration, based on posted measures of voluntary compliance with data sharing restrictions.
to:

!!! Real-time collaborative editing
Real-time collaborative editing including an insightful understanding of real-time requirements for collaborative editing, grounded in a theory for the effect of real-time constraints in collaborative
work. Current related work is fundamentally flawed, based on tasks with varied time constants, idiosyncratic task coupling and uncontrolled compensatory strategies. The project focusses as well on awareness management for the coordination of work in the presence of conflict and disruption.

!!! Trust-based collaboration
Trust-based collaboration including validation of proposed light security mechanisms for decentralised collaboration, based on posted measures of voluntary compliance with data sharing restrictions.

!!! Validation
Changed line 11 from:
We have studied real-time requirements in collaborative editing. We examined user behavior of 80 participants organized into teams of four, conducting three different editing tasks using levels of delay ranging between 0 and 10 seconds of delay in the distribution of document changes across users: 
to:
We have studied real-time requirements in collaborative editing. We examined user behavior of 80 participants organized into teams of four, conducting three different editing tasks using levels of delay ranging between 0 and 10 seconds of delay in the distribution of document changes across users:
Changed lines 4-5 from:

*''Real-time collaborative editing'' including an insightful understanding of real-time requirements for collaborative editing, grounded in a theory for the effect of real-time constraints in collaborative work. Current related work is fundamentally flawed, based on tasks with varied time constants, idiosyncratic task coupling and uncontrolled compensatory strategies. The project focusses as well on awareness management for the coordination of work in the presence of conflict and disruption.
to:
*Real-time collaborative editing including an insightful understanding of real-time requirements for collaborative editing, grounded in a theory for the effect of real-time constraints in collaborative work. Current related work is fundamentally flawed, based on tasks with varied time constants, idiosyncratic task coupling and uncontrolled compensatory strategies. The project focusses as well on awareness management for the coordination of work in the presence of conflict and disruption.
Changed line 19 from:
* ''What is the effect of delay on task process and outcome?''
to:
!!! What is the effect of delay on task process and outcome?
Changed line 22 from:
* ''What is the effect of strategy on task process and outcome?''
to:
!!! What is the effect of strategy on task process and outcome?
Changed line 25 from:
* ''How does strategy interact with delay to affect task process and outcome?''
to:
!!! How does strategy interact with delay to affect task process and outcome?
Changed line 5 from:
%link-list% *''Real-time collaborative editing'' including an insightful understanding of real-time requirements for collaborative editing, grounded in a theory for the effect of real-time constraints in collaborative work. Current related work is fundamentally flawed, based on tasks with varied time constants, idiosyncratic task coupling and uncontrolled compensatory strategies. The project focusses as well on awareness management for the coordination of work in the presence of conflict and disruption.
to:
*''Real-time collaborative editing'' including an insightful understanding of real-time requirements for collaborative editing, grounded in a theory for the effect of real-time constraints in collaborative work. Current related work is fundamentally flawed, based on tasks with varied time constants, idiosyncratic task coupling and uncontrolled compensatory strategies. The project focusses as well on awareness management for the coordination of work in the presence of conflict and disruption.
Changed line 5 from:
*%link-list%''Real-time collaborative editing'' including an insightful understanding of real-time requirements for collaborative editing, grounded in a theory for the effect of real-time constraints in collaborative work. Current related work is fundamentally flawed, based on tasks with varied time constants, idiosyncratic task coupling and uncontrolled compensatory strategies. The project focusses as well on awareness management for the coordination of work in the presence of conflict and disruption.
to:
%link-list% *''Real-time collaborative editing'' including an insightful understanding of real-time requirements for collaborative editing, grounded in a theory for the effect of real-time constraints in collaborative work. Current related work is fundamentally flawed, based on tasks with varied time constants, idiosyncratic task coupling and uncontrolled compensatory strategies. The project focusses as well on awareness management for the coordination of work in the presence of conflict and disruption.
Changed lines 5-6 from:
%class=link-list%
*
''Real-time collaborative editing'' including an insightful understanding of real-time requirements for collaborative editing, grounded in a theory for the effect of real-time constraints in collaborative work. Current related work is fundamentally flawed, based on tasks with varied time constants, idiosyncratic task coupling and uncontrolled compensatory strategies. The project focusses as well on awareness management for the coordination of work in the presence of conflict and disruption.
to:
*%link-list%''Real-time collaborative editing'' including an insightful understanding of real-time requirements for collaborative editing, grounded in a theory for the effect of real-time constraints in collaborative work. Current related work is fundamentally flawed, based on tasks with varied time constants, idiosyncratic task coupling and uncontrolled compensatory strategies. The project focusses as well on awareness management for the coordination of work in the presence of conflict and disruption.
Changed lines 5-6 from:
*%class=link-list%''Real-time collaborative editing'' including an insightful understanding of real-time requirements for collaborative editing, grounded in a theory for the effect of real-time constraints in collaborative work. Current related work is fundamentally flawed, based on tasks with varied time constants, idiosyncratic task coupling and uncontrolled compensatory strategies. The project focusses as well on awareness management for the coordination of work in the presence of conflict and disruption.
to:
%class=link-list%
*
''Real-time collaborative editing'' including an insightful understanding of real-time requirements for collaborative editing, grounded in a theory for the effect of real-time constraints in collaborative work. Current related work is fundamentally flawed, based on tasks with varied time constants, idiosyncratic task coupling and uncontrolled compensatory strategies. The project focusses as well on awareness management for the coordination of work in the presence of conflict and disruption.
Changed lines 4-5 from:
>>font-style:bold<<
*
''Real-time collaborative editing'' including an insightful understanding of real-time requirements for collaborative editing, grounded in a theory for the effect of real-time constraints in collaborative work. Current related work is fundamentally flawed, based on tasks with varied time constants, idiosyncratic task coupling and uncontrolled compensatory strategies. The project focusses as well on awareness management for the coordination of work in the presence of conflict and disruption.
to:

*%class=link-list%''Real-time collaborative editing'' including an insightful understanding of real-time requirements for collaborative editing, grounded in a theory for the effect of real-time constraints in collaborative work. Current related work is fundamentally flawed, based on tasks with varied time constants, idiosyncratic task coupling and uncontrolled compensatory strategies. The project focusses as well on awareness management for the coordination of work in the presence of conflict and disruption.
Added line 4:
>>font-style:bold<<
Changed line 11 from:
We have examined user behavior of 80 participants organized into teams of four, conducting three different editing tasks using levels of delay ranging between 0 and 10 seconds of delay in the distribution of document changes across users: 
to:
We have studied real-time requirements in collaborative editing. We examined user behavior of 80 participants organized into teams of four, conducting three different editing tasks using levels of delay ranging between 0 and 10 seconds of delay in the distribution of document changes across users: 
Changed line 11 from:
We have examined user behavior of 60 participants organized into teams of four, conducting three different editing tasks using levels of delay ranging between 0 and 10 seconds of delay in the distribution of document changes across users: 
to:
We have examined user behavior of 80 participants organized into teams of four, conducting three different editing tasks using levels of delay ranging between 0 and 10 seconds of delay in the distribution of document changes across users: 
Changed lines 26-43 from:
The effect of delay depends on strategy. However, somewhat counterintuitively, and unlike previous research, the overall superior strategy does not overcome the effect of delay.  In fact, the insertion sort score declined with delay for continuous sort, but did not for sort-at-the-end.  We suspect that continuous sort entails more coupling, because years must be in place prior to positioning, and because text position is changing frequently throughout the entire task as sorting proceeds.  Even continuous-sort participants in the 0  and 4 second delay conditions complained about the ``jumping'' line positions.  To manage the coupling in continuous sort, we see participants slow down with delay. However, the negative slope on the insertion sort metric for continuous sort relative to the flat slope for sort-at-the-end suggests that the continuous sort strategy is only adaptive within a range of delay. Untested levels of delay could actually result in worse performance for continuous sorting than a sort-at-the-end strategy.  Furthermore, the chat metric suggests that sort-at-the-end requires more local coordination as delay increases. Thus the coordination established by formal agreement at the outset in continuous-sort appears to favor efficient communication over the ability to respond to local perturbations.  On the other hand, sort-at-the-end appears to favor the ability to respond to local perturbations at the expense of efficient communication. 


!!Experimentations of real-time editing constraints
[[#ExperimentationsRTEC]]

* April 18, 2013, 10h-11h30, salle PO, TELECOM Nancy (Investigators: François CHAROY, Gérald OSTER)
* April 18, 2013, 11h30-13h, salle PO, TELECOM Nancy (Investigators: François CHAROY, Gérald OSTER)
* April 22, 2013, 14h-15h30, salle PO, TELECOM Nancy (Investigators: François CHAROY, Gérald OSTER)
* May 24, 2013, 10h-11h30, salle PO, TELECOM Nancy (Investigators: Valerie SHALIN, Claudia IGNAT, Gérald OSTER)
* May 27, 2013, 11h-12h30, salle PO, TELECOM Nancy (Investigators: Valerie SHALIN, Claudia IGNAT, Gérald OSTER)
* May 28, 2013, 10h-11h30, salle PO, TELECOM Nancy (Investigators: Valerie SHALIN, Claudia IGNAT, Gérald OSTER)
* May 28, 2013, 14h-15h30, salle PO, TELECOM Nancy (Investigators: Valerie SHALIN, Claudia IGNAT, Gérald OSTER)
* May 31, 2013, 11h-12h30, salle PO, TELECOM Nancy (Investigators: Valerie SHALIN, Claudia IGNAT, Gérald OSTER)
* July 2, 2013, 14h-15h30, salle PO, TELECOM Nancy (Investigators: Claudia IGNAT, Gérald OSTER)
* July 4, 2013, 10h-11h30, salle PO, TELECOM Nancy (Investigators: Claudia IGNAT, Gérald OSTER)
* July 5, 2013, 10h-11h30, salle PO, TELECOM Nancy (Investigators: Claudia IGNAT, Gérald OSTER)
* July 7, 2013, 10h-11h30, salle PO, TELECOM Nancy (Investigators: Claudia IGNAT, Gérald OSTER)
to:
The effect of delay depends on strategy. However, somewhat counterintuitively, and unlike previous research, the overall superior strategy does not overcome the effect of delay.  In fact, the insertion sort score declined with delay for continuous sort, but did not for sort-at-the-end.  We suspect that continuous sort entails more coupling, because years must be in place prior to positioning, and because text position is changing frequently throughout the entire task as sorting proceeds.  Even continuous-sort participants in the 0  and 4 second delay conditions complained about the ``jumping'' line positions.  To manage the coupling in continuous sort, we see participants slow down with delay. However, the negative slope on the insertion sort metric for continuous sort relative to the flat slope for sort-at-the-end suggests that the continuous sort strategy is only adaptive within a range of delay. Untested levels of delay could actually result in worse performance for continuous sorting than a sort-at-the-end strategy.  Furthermore, the chat metric suggests that sort-at-the-end requires more local coordination as delay increases. Thus the coordination established by formal agreement at the outset in continuous-sort appears to favor efficient communication over the ability to respond to local perturbations.  On the other hand, sort-at-the-end appears to favor the ability to respond to local perturbations at the expense of efficient communication. 
Changed lines 4-6 from:
*"Real-time collaborative editing" including an insightful understanding of real-time requirements for collaborative editing, grounded in a theory for the effect of real-time constraints in collaborative work. Current related work is fundamentally flawed, based on tasks with varied time constants, idiosyncratic task coupling and uncontrolled compensatory strategies. The project focusses as well on awareness management for the coordination of work in the presence of conflict and disruption.
*"Trust-based collaboration"  including validation of proposed light security mechanisms for decentralised collaboration, based on posted measures of voluntary compliance with data sharing restrictions.
to:
*''Real-time collaborative editing'' including an insightful understanding of real-time requirements for collaborative editing, grounded in a theory for the effect of real-time constraints in collaborative work. Current related work is fundamentally flawed, based on tasks with varied time constants, idiosyncratic task coupling and uncontrolled compensatory strategies. The project focusses as well on awareness management for the coordination of work in the presence of conflict and disruption.
*''Trust-based collaboration''  including validation of proposed light security mechanisms for decentralised collaboration, based on posted measures of voluntary compliance with data sharing restrictions.
Changed lines 19-20 from:
* "What is the effect of delay on task process and outcome?"
to:
* ''What is the effect of delay on task process and outcome?''
Changed lines 22-23 from:
* "What is the effect of strategy on task process and outcome?"
to:
* ''What is the effect of strategy on task process and outcome?''
Changed lines 25-26 from:
* "How does strategy interact with delay to affect task process and outcome?"
to:
* ''How does strategy interact with delay to affect task process and outcome?''
Added lines 8-30:

!! Progress and ongoing work

We have examined user behavior of 60 participants organized into teams of four, conducting three different editing tasks using levels of delay ranging between 0 and 10 seconds of delay in the distribution of document changes across users: 
* A proofreading task, in which participants corrected a short text, containing several grammatical and spelling errors
* A sorting task, in which participants located the release dates of an alphabetized list of movies, and sorted them accordingly and
* A note taking task, in which participants listed to a 10 minute interview on the topic of cloud computing, and provided an integrated set of notes on the interview
* All participants completed a follow-up questionnaire at the completion of the three task series. 

Analysis thus far has focused on the sorting task, using measures of sorting accuracy based on the insertion sort algorithm, average time per entry, chat behavior, collisions between users, and crucially, task strategy. We note here some key findings:

* "What is the effect of delay on task process and outcome?"

The general effect of delay is to slow down performance, which in this case hinders task outcome. Second, we suggest that the effect delay is a function of the task time constant. In our case, with a task time constant on the order of 3.5 seconds, the effective manipulations are much larger than the 700 msec characteristic of motor control tasks, with problems becoming apparent at 8 and 10 seconds.  Third, the effect of a system property such as delay can be small relative to strategies. Fourth, the effect of delay need not be linear.  Complaint about delay was a low frequency event at the level of individual participants.  This makes sense because participants would have had to experience the direct consequence of delay in order to detect it.  Collisions are one such consequence, which depending upon strategy, participants did not always recognize and repair. Nevertheless, at the group level, most groups had at least one participant complain about delay at the highest levels. 

* "What is the effect of strategy on task process and outcome?"

The relationship between speed and insertion sort scores, is not in general a speed accuracy tradeoff.  Second, a strategy effect appears in the outcome measure, response time,  and interface ratings. Groups that sorted continuously (relative to those who postponed sorting) liked the interface better than sort at the end groups, perhaps reflecting satisfaction with the higher level of task success. 

* "How does strategy interact with delay to affect task process and outcome?"

The effect of delay depends on strategy. However, somewhat counterintuitively, and unlike previous research, the overall superior strategy does not overcome the effect of delay.  In fact, the insertion sort score declined with delay for continuous sort, but did not for sort-at-the-end.  We suspect that continuous sort entails more coupling, because years must be in place prior to positioning, and because text position is changing frequently throughout the entire task as sorting proceeds.  Even continuous-sort participants in the 0  and 4 second delay conditions complained about the ``jumping'' line positions.  To manage the coupling in continuous sort, we see participants slow down with delay. However, the negative slope on the insertion sort metric for continuous sort relative to the flat slope for sort-at-the-end suggests that the continuous sort strategy is only adaptive within a range of delay. Untested levels of delay could actually result in worse performance for continuous sorting than a sort-at-the-end strategy.  Furthermore, the chat metric suggests that sort-at-the-end requires more local coordination as delay increases. Thus the coordination established by formal agreement at the outset in continuous-sort appears to favor efficient communication over the ability to respond to local perturbations.  On the other hand, sort-at-the-end appears to favor the ability to respond to local perturbations at the expense of efficient communication. 

Added lines 8-23:

!!Experimentations of real-time editing constraints
[[#ExperimentationsRTEC]]

* April 18, 2013, 10h-11h30, salle PO, TELECOM Nancy (Investigators: François CHAROY, Gérald OSTER)
* April 18, 2013, 11h30-13h, salle PO, TELECOM Nancy (Investigators: François CHAROY, Gérald OSTER)
* April 22, 2013, 14h-15h30, salle PO, TELECOM Nancy (Investigators: François CHAROY, Gérald OSTER)
* May 24, 2013, 10h-11h30, salle PO, TELECOM Nancy (Investigators: Valerie SHALIN, Claudia IGNAT, Gérald OSTER)
* May 27, 2013, 11h-12h30, salle PO, TELECOM Nancy (Investigators: Valerie SHALIN, Claudia IGNAT, Gérald OSTER)
* May 28, 2013, 10h-11h30, salle PO, TELECOM Nancy (Investigators: Valerie SHALIN, Claudia IGNAT, Gérald OSTER)
* May 28, 2013, 14h-15h30, salle PO, TELECOM Nancy (Investigators: Valerie SHALIN, Claudia IGNAT, Gérald OSTER)
* May 31, 2013, 11h-12h30, salle PO, TELECOM Nancy (Investigators: Valerie SHALIN, Claudia IGNAT, Gérald OSTER)
* July 2, 2013, 14h-15h30, salle PO, TELECOM Nancy (Investigators: Claudia IGNAT, Gérald OSTER)
* July 4, 2013, 10h-11h30, salle PO, TELECOM Nancy (Investigators: Claudia IGNAT, Gérald OSTER)
* July 5, 2013, 10h-11h30, salle PO, TELECOM Nancy (Investigators: Claudia IGNAT, Gérald OSTER)
* July 7, 2013, 10h-11h30, salle PO, TELECOM Nancy (Investigators: Claudia IGNAT, Gérald OSTER)
Changed lines 1-12 from:
This project focuses on the human evaluation of the SCORE team’s computer science algorithms for trustworthy collaborative editing. The project enhances the acquisition of in-house expertise with the inclusion of researchers from the U.S. who provide expertise in user studies.


Technical challenges

The project focuses on two substantive areas (1a
and 1b), with one overarching methodological contribution (2).

1a. Real-time requirements for collaborative editing, grounded in a theory
for the effect of real-time constraints in collaborative work. Current related work is fundamentally flawed, based on tasks with varied time constants, idiosyncratic task coupling and uncontrolled compensatory strategies.

1b. Trust policies in collaborative editing including awareness management for the coordination of work in
the presence of conflict and disruption. The project includes also validation of proposed light security mechanisms for decentralised collaboration, based on posted measures of voluntary compliance with data sharing restrictions.

2.
Methodologically, validation requires the expertise of both computer scientists that designed the systems and social scientists for conceptualizing and measuring human behaviour in collaborative work. We are developing new methods for the cost-effective evaluation of collaborative work to compensate for otherwise unrealistic sample sizes and costly engineering, using game theory to inspire task analogues and simulated users along with human users.
to:
This project focuses on the human evaluation of methods and algorithms for trustworthy collaborative editing. The project brings together expertise on distributed collaborative systems of SCORE team with expertise in user studies of Department of Psychology of Wright State University.

The project focuses on two substantive areas - real-time collaborative editing
and trust-based collaboration-, with one overarching methodological contribution.
*"Real-time collaborative editing" including an insightful understanding of real-time requirements for collaborative editing, grounded in a theory for the effect of real-time constraints in collaborative work. Current related work is fundamentally flawed, based on tasks with varied time constants, idiosyncratic task coupling and uncontrolled compensatory strategies. The project focusses as well on awareness management for the coordination of work in the presence of conflict and disruption.
*"Trust-based collaboration"  including validation of proposed light security mechanisms for decentralised collaboration, based on posted measures of voluntary compliance with data sharing restrictions.

Methodologically, validation requires the expertise of both computer scientists that designed the systems and social scientists for conceptualizing and measuring human behaviour in collaborative work. We are developing new methods for the cost-effective evaluation of collaborative work to compensate for otherwise unrealistic sample sizes and costly engineering, using game theory to inspire task analogues and simulated users along with human users.
Added lines 1-12:
This project focuses on the human evaluation of the SCORE team’s computer science algorithms for trustworthy collaborative editing. The project enhances the acquisition of in-house expertise with the inclusion of researchers from the U.S. who provide expertise in user studies.


Technical challenges

The project focuses on two substantive areas (1a and 1b), with one overarching methodological contribution (2).

1a. Real-time requirements for collaborative editing, grounded in a theory for the effect of real-time constraints in collaborative work. Current related work is fundamentally flawed, based on tasks with varied time constants, idiosyncratic task coupling and uncontrolled compensatory strategies.

1b. Trust policies in collaborative editing including awareness management for the coordination of work in the presence of conflict and disruption. The project includes also validation of proposed light security mechanisms for decentralised collaboration, based on posted measures of voluntary compliance with data sharing restrictions.

2. Methodologically, validation requires the expertise of both computer scientists that designed the systems and social scientists for conceptualizing and measuring human behaviour in collaborative work. We are developing new methods for the cost-effective evaluation of collaborative work to compensate for otherwise unrealistic sample sizes and costly engineering, using game theory to inspire task analogues and simulated users along with human users.