curate.py: default GPU a10g-small (l4x1 often slow to schedule)
Browse files
README.md
CHANGED
|
@@ -76,7 +76,7 @@ hf jobs uv run curate.py -- --dataset HuggingFaceFW/fineweb-edu --config sample-
|
|
| 76 |
--limit 10000 --out-repo YOUR_NAME/fineweb-edu-curated
|
| 77 |
```
|
| 78 |
|
| 79 |
-
The script's PEP 723 header carries the Job settings (`[tool.hf-jobs]`: `
|
| 80 |
to push the result), so the command needs only the script's own arguments. Flags such as `--flavor rtx-pro-6000`
|
| 81 |
override the header.
|
| 82 |
|
|
|
|
| 76 |
--limit 10000 --out-repo YOUR_NAME/fineweb-edu-curated
|
| 77 |
```
|
| 78 |
|
| 79 |
+
The script's PEP 723 header carries the Job settings (`[tool.hf-jobs]`: `a10g-small` GPU, 1-hour timeout, your `HF_TOKEN`
|
| 80 |
to push the result), so the command needs only the script's own arguments. Flags such as `--flavor rtx-pro-6000`
|
| 81 |
override the header.
|
| 82 |
|
curate.py
CHANGED
|
@@ -5,7 +5,7 @@
|
|
| 5 |
#
|
| 6 |
# # Defaults for `hf jobs uv run`; command-line flags override them.
|
| 7 |
# [tool.hf-jobs]
|
| 8 |
-
# flavor = "
|
| 9 |
# timeout = "1h"
|
| 10 |
# secrets = ["HF_TOKEN"]
|
| 11 |
# ///
|
|
|
|
| 5 |
#
|
| 6 |
# # Defaults for `hf jobs uv run`; command-line flags override them.
|
| 7 |
# [tool.hf-jobs]
|
| 8 |
+
# flavor = "a10g-small"
|
| 9 |
# timeout = "1h"
|
| 10 |
# secrets = ["HF_TOKEN"]
|
| 11 |
# ///
|