Click the blue text to follow immediately
Why Do We Need DACP+SDF Instead of HTTP+HTML?
One-sentence answer:HTTP+HTML allows people to “connect to” web pages, while DACP+SDF enables data to “connect tightly” on its own—not only can it navigate, but it can also compute on-site, stream back in seconds, and grow automatically. This is how a scientific data network should look.
01First, ask three soul-searching questions
| Scenario | Typical experience with HTTP+HTML | Real needs of scientists |
| Finding data | Scraping web pages, clicking directories, downloading entire packages | Locating to columns, rows, and time slices in seconds |
| Reading data | Download → Unzip → Convert format → Adjust library | Open as DataFrame, zero parsing |
| Using data | Calculate after moving, re-download from breakpoints | Calculate on-site, only pull back results, use while calculating |
Conclusion:HTTP excels at transmitting documents but is not good at “transmitting tables + algorithms”; HTML is for human reading, not for Spark or Pandas. We need a protocol stack that is “table-native, navigable, and computable”—DACP+SDF.
02What are DACP+SDF?
-
DACP: Unified addressing, authorization, reverse supply (COOK)
-
SDF (Streaming Data Frame): Two-dimensional table + columnar compression + streaming transmission + cell hyperlinks (Ref)
In one sentence:SDF is the “HTML” of scientific data, DACP is the “HTTP” of scientific data—but designed specifically for machines..
03Comparison: HTTP+HTML vs. DACP+SDF
| Dimension | HTTP+HTML | DACP+SDF |
| Basic unit | Document/URL | Two-dimensional table/URL |
| Association method | <a href=””> Human clicks | Ref cell, machine jumps |
| Reading cost | Parse DOM → Regex → Type conversion | Direct DataFrame, zero parsing |
| Calculation location | Client moves complete files | On-site COOK, only return results |
| Compression ratio | Text + tag redundancy | Columnar + dictionary, 3-5 times improvement |
| Streaming | Chunk segmentation, unreadable by humans | Row streaming frames, calculations start in seconds |
| Toolchain | Browser, crawler | Native to Spark, Pandas, NumPy |
04Association capability: HTML allows human navigation, SDF allows data navigation
SDF cells can embed Ref:
| station_id | temp | next_station |
| A0101 | 28.3 | dacp://noaa/wave#A0102 |
The machine only needs to execute df.next_station.read() toseamlessly jump to another table—data weaves its own network, without human clicks. This is exactly the Linked Data concept proposed by Tim Berners-Lee, but SDF replaces RDF triples with two-dimensional tables,with zero friction with existing big data stacks.
05On-site computation: HTTP moves files, DACP moves algorithms
HTTP paradigm:GET → Download → ComputeDACP paradigm:COOK Recipe → Node computation → Result streams back
-
Network traffic decreases by 100-1000 times
-
Computation delay reduced from “hours” to “minutes” or even “seconds”
-
No more need to “fill up the hard drive before cleaning”
06Ecological compatibility: RDF is philosophy, SDF is engineering
| Feature | HTTP+RDF | DACP+SDF |
| Knowledge expression | Triples, strong inference | Two-dimensional tables, strong statistics |
| Toolchain | Graph databases, SPARQL | Spark, Pandas, NumPy |
| Entry barrier | Requires OWL thinking | Only needs to know DataFrame |
| With big data | Requires ETL conversion | Native zero conversion |
Conclusion:RDF is suitable for the “semantic web”, while SDF is suitable for the “data factory”—scientific computing precisely needs the latter..
07Network effect: Every new node creates a new highway intersection in the world
-
Node as entry:Deploying a daemon ≈ Hanging a “highway sign”
-
Result as entry:Output stream frames also have DACP URLs, which can be COOKed again
-
Permission as entry:One-time Token + sandbox, making “daring to open” the default
Every time a data center goes online, the total network value increases by 1; every time a result is produced, a potential entry increases by 1. This is the first implementation of Metcalfe’s law in the field of scientific data.
08In conclusion: A protocol, a network
HTTP+HTML enables human connectivity;
DACP+SDF enables data connectivity—discoverable, navigable, computable, and growable.
Therefore, what we need is not a “larger hard drive”, but a “smarter network”.
With DACP+SDF, the scientific data network is truly born.
09Preview · Next Article
“DACP Protocol Part Three: Scientific Data Collaboration Protocol”, we will upgrade the “single-point kitchen” to a “central kitchen”: how can a Recipe fire across multiple nodes in parallel? How to synchronize versions and schedule computing power across centers? Stay tuned!
ENDRECOMMENDPrevious Recommendations
DACP Protocol Part One: What is the Scientific Data Access Protocol?
DACP Protocol Part Two: Scientific Data Reverse Supply Protocol
