Skip to content

added DVC.org to existing tools #8

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 2 commits into
base: master
Choose a base branch
from
Open

added DVC.org to existing tools #8

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
63558d4
Update README.md
Jun 8, 2019
64cb823
changed links for learning DVC
Jun 9, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,8 @@ But, UNF is not perfect. The problems include:
- It is quite sensitive to data structure (e.g., "wide" and "long" representations of the same dataset produce different UNFs)
- It is not a version control system and provides essentially no insights into what changed, only that a change occurred

[DVC.org](https://github.com/iterative/dvc) is a data-versioning tool which is almost similar to working of Git-SCM. You can learn about it through [this post](https://dvc.org/doc/get-started) and [this](https://dvc.org/doc/tutorial).

All of these tools also focus on the data themselves, rather than associated metadata (e.g., the codebook describing the data). While some data formats (e.g., proprietary formats like Stata's .dta and SPSS's .sav) encode this metadata directly in the file, it is not a common feature of widely text-delimited data structures. Sometimes codebooks are modified independent of data values and vice versa, but it's rather to see large public datasets provide detailed information about changes to either the data or the codebook, except in occasional releases.

Another major challenge to data versioning is that existing tools version control are not well-designed to handle provenance. When data is generated, stored, or modified, a software-oriented version control system has no obvious mechanism for recording *why* values in a dataset are what they are or why changes are made to particular values. A commit message might provide this information, but as soon as a value is changed again, the history of changes *to a particular value* are lost in the broader history of the data file as a whole.
Expand Down