Comments on: Copy-On-Write 101 – Part 3: The Sum of Its Parts http://hackerboss.com/copy-on-write-101-part-3-the-sum-of-its-parts/ Developing software and managing development teams. Sun, 08 Aug 2010 20:15:47 +0000 http://wordpress.org/?v= hourly 1 By: Ville Laurikarihttp://hackerboss.com/copy-on-write-101-part-3-the-sum-of-its-parts/comment-page-1/#comment-212 Ville Laurikari Tue, 11 Aug 2009 14:12:21 +0000 http://hackerboss.com/?p=533#comment-212 @axltoth, thanks. You're not the only one to mention Clojure (see <a href="http://hackerboss.com/i-have-seen-the-future-and-it-is-copy-on-write/#comment-109" rel="nofollow">here</a>). Clojure doesn't have automatically disk-backed data structures, Shark does. On the other hand, Shark doesn't exist... so I guess Clojure wins, eh? @axltoth, thanks. You’re not the only one to mention Clojure (see here). Clojure doesn’t have automatically disk-backed data structures, Shark does. On the other hand, Shark doesn’t exist… so I guess Clojure wins, eh?

]]> By: alxtothhttp://hackerboss.com/copy-on-write-101-part-3-the-sum-of-its-parts/comment-page-1/#comment-194 alxtoth Mon, 10 Aug 2009 20:07:54 +0000 http://hackerboss.com/?p=533#comment-194 Hi,Apparently you like LISP-like languages . Check out Clojure (<a href="http://www.clojure.org/" rel="nofollow">www.clojure.org</a>), it has "transactional memory"-Alex Hi,

Apparently you like LISP-like languages . Check out Clojure (http://www.clojure.org), it has “transactional memory”

-Alex

]]> By: Ville Laurikarihttp://hackerboss.com/copy-on-write-101-part-3-the-sum-of-its-parts/comment-page-1/#comment-128 Ville Laurikari Thu, 23 Jul 2009 11:31:18 +0000 http://hackerboss.com/?p=533#comment-128 All data structures in Shark are copy-on-write. Because existing data structures are never modified in-place, everything is automatically thread safe. So yes, the data structure for the STM database is thread safe, like all other data structures in Shark.Conflict detection only happens when replaying the log. The worker threads are oblivious to the fact that other worker threads may be concurrently using the same database. They'll find out later if there was a conflict. All data structures in Shark are copy-on-write. Because existing data structures are never modified in-place, everything is automatically thread safe. So yes, the data structure for the STM database is thread safe, like all other data structures in Shark.

Conflict detection only happens when replaying the log. The worker threads are oblivious to the fact that other worker threads may be concurrently using the same database. They’ll find out later if there was a conflict.

]]> By: Vagn Johansenhttp://hackerboss.com/copy-on-write-101-part-3-the-sum-of-its-parts/comment-page-1/#comment-125 Vagn Johansen Thu, 23 Jul 2009 08:04:32 +0000 http://hackerboss.com/?p=533#comment-125 Yes, almost. The explanation of the replay log explains why you call it an STM database. It is like a small program that can be replayed/re-run when a conlict is detected.The way you describe it, the owner thread can update the database and at the same time a worker thread can traverse the database (via a different snapshot). Being a copy-on-write data structure the owner thread can delete a node that the worker thread is trying to traverse due to the node sharing (assuming something like the tree in part1).I assume this means that the data structure for the STM database has some form of thread-safety built-in? And regarding the conflict detection does it happens both during replays on the owner thread and during traversals in the worker thread? Yes, almost. The explanation of the replay log explains why you call
it an STM database. It is like a small program that can be
replayed/re-run when a conlict is detected.

The way you describe it, the owner thread can update the database and
at the same time a worker thread can traverse the database (via a
different snapshot). Being a copy-on-write data structure the owner
thread can delete a node that the worker thread is trying to traverse
due to the node sharing (assuming something like the tree in part1).

I assume this means that the data structure for the STM database has
some form of thread-safety built-in? And regarding the conflict detection
does it happens both during replays on the owner thread and during
traversals in the worker thread?

]]> By: Ville Laurikarihttp://hackerboss.com/copy-on-write-101-part-3-the-sum-of-its-parts/comment-page-1/#comment-122 Ville Laurikari Wed, 22 Jul 2009 19:05:01 +0000 http://hackerboss.com/?p=533#comment-122 Vagn, that paragraph could have been more clear, for sure. I'll try to clarify how STM works with copy-on-write data structures.For an STM database there is one "owner" thread which keeps the latest snapshot of the database. Other "worker" threads can request the latest snapshot to work on. They do this by sending a message to the owner which then replies back with the snapshot.The worker thread then goes off an does whatever it wants to with the database, possibly updating it. When it has finished with the transaction, the worker sends its modified database back to the owner, along with a log of all reads and writes it performed.The owner thread will then replay the log with the <i>actual</i> latest database snapshot. If there are no conflicts, the owner replaces the latest snapshot with the one resulting from replaying the log. If there are conflicts, the owner keeps the old snapshot and thus discards all updates from the worker. The worker thread can then retry the transaction again with the new latest snapshot.The owner thread can take some shortcuts. If there were no other concurrent updates, it can directly accept the new database snapshot received from the worker. If the worker did not do any writes there is no need for conflict resolution.Does this answer your question? Vagn, that paragraph could have been more clear, for sure. I’ll try
to clarify how STM works with copy-on-write data structures.

For an STM database there is one “owner” thread which keeps the
latest snapshot of the database. Other “worker” threads can request
the latest snapshot to work on. They do this by sending a message to
the owner which then replies back with the snapshot.

The worker thread then goes off an does whatever it wants to with the
database, possibly updating it. When it has finished with the
transaction, the worker sends its modified database back to the owner,
along with a log of all reads and writes it performed.

The owner thread will then replay the log with the actual latest
database snapshot. If there are no conflicts, the owner replaces
the latest snapshot with the one resulting from replaying the log. If
there are conflicts, the owner keeps the old snapshot and thus
discards all updates from the worker. The worker thread can then
retry the transaction again with the new latest snapshot.

The owner thread can take some shortcuts. If there were no other
concurrent updates, it can directly accept the new database snapshot
received from the worker. If the worker did not do any writes there is no need for conflict resolution.

Does this answer your question?

]]> By: Vagn Johansenhttp://hackerboss.com/copy-on-write-101-part-3-the-sum-of-its-parts/comment-page-1/#comment-120 Vagn Johansen Wed, 22 Jul 2009 15:56:51 +0000 http://hackerboss.com/?p=533#comment-120 I don't understand the paragraph with "The natural model..". How does this happen "concurrently"? Via messages to the owner thread? I don’t understand the paragraph with “The natural model..”. How does this happen “concurrently”? Via messages to the owner thread?

]]>