Shorter model card in the original format

README.md

@@ -1,7 +1,5 @@
 ---
 license: cc-by-4.0
-language:
-  - en
 tags:
   - reinforcement-learning
   - multi-objective
@@ -9,106 +7,58 @@ tags:
   - attention
   - ppo
   - morl
+language:
+  - en
 pipeline_tag: reinforcement-learning
 ---
 
 # SmartFlexPack MORL: Sequential Attention Policy for Packaging Scheduling
 
-Five trained PyTorch policies accompanying the manuscript *Multi-Objective
-Reinforcement Learning for Adaptive Packaging Scheduling via Sequential Attention
-Policy*, by L. Venturoso, G. Egger, F. Visentin and M. Todescato (2026,
-manuscript in preparation). Developed within the SMARTFLEXPACK project at
-Fraunhofer Italia.
+Trained model weights for the paper:
 
-## Model and task
+> **Multi-Objective Reinforcement Learning for Adaptive Packaging Scheduling via Sequential Attention Policy**  
+> L. Venturoso, G. Egger, F. Visentin, M. Todescato  
+> Fraunhofer Italia Research EFRE1054 Smartflexpack  
+> *Manuscript in preparation*
 
-The policy chooses INLINE printing or LABEL packaging for each order when both
-modes are feasible. A preference vector `(tardiness, cost, CO2)`, with nonnegative
-entries summing to one, controls the trade-off at inference time.
-The planner handles order sequencing, forced modes including NO_PRINT, line
-assignment, timing and label reservations; the discrete-event simulation executes
-the resulting production plan.
+## Model description
 
-`SeqAttentionActorCritic` has **12,355 parameters**:
+`SeqAttentionActorCritic` (12,355 parameters) decides, for each order, between
+inline printing and labelling. At each planning event a one-layer multi-head
+self-attention encoder (4 heads, d=32) encodes the pending orders; for each
+decision a 54→64→64 MLP combines that embedding with ten live decision features
+and twelve global features, including the preference vector
+`(tardiness, cost, CO2)`, and outputs two action logits and a value.
 
-- Once per planning event: five features per pending order are embedded into
-  32 dimensions, with sinusoidal due-date-rank encoding and one self-attention
-  layer (four heads, residual connection and LayerNorm).
-- Once per decision: the 32-dimensional embedding is concatenated with ten live
-  decision features and twelve global features, including the three preferences.
-  A `54 → 64 → 64` tanh MLP feeds two action logits and a scalar critic.
+A single trained policy covers the whole trade-off: the operator sets the
+preference at inference time, with no retraining.
 
-The encoder processes the orders pending at that event. The 600 orders in a
-training episode arrive across multiple planning events.
+## Results
 
-## Training
+In the reference plant configuration (3 label lines, inline 0.45 s/pack) the
+policy exceeds the hypervolume of the full family of 32 realisable reference
+controllers by **+0.058** (95% CI [+0.025, +0.095], about 7%), replicated on an
+independent seed set (+0.050). The advantage grows with a slower inline line
+(12–15% at 0.60 s/pack); with a single label line, and with the fastest inline
+line, the policy ties the reference family or falls slightly below it.
 
-Behavioural cloning provides the warm start (240 demonstration episodes,
-40 epochs), followed by five independent PPO runs, each with 200 updates,
-ten parallel episodes per update and 600 orders per episode. Adam uses a learning
-rate of `3e-4`, clipping `0.2`, three PPO epochs, entropy coefficient `0.01` and
-value-loss coefficient `0.5`. The critic target is the final reward of the
-individual order, and advantages are batch-standardised.
-
-Preferences remain fixed within an episode. Half are sampled from
-Dirichlet(1,1,1); half favour a randomly selected objective with weight sampled
-uniformly from 0.6 to 0.9. Training varies the label-line count from one to five,
-with one inline line at 0.35 s/pack and a consolidating label-line rule.
-
-Rewards combine clipped, normalised tardiness, cost and emissions penalties,
-plus a 0.5 penalty for late DDP/DAP orders. Each order's cost includes an equal
-share of the episode's line operating cost. Reward normalisers are provided
-in `normalisation/reward_normalisers_consol.json`; observation transforms are
-implemented in the full project's `experiments/seq_morl/envs/observations.py`.
-
-These are the final historical paper checkpoints, with identical tensors to
-update 200. The supplied configuration and inference source reflect the current
-code. Current code also seeds BC initialization and orders parallel collection;
-retraining can differ from the historical runs. These weight files contain no
-optimizer or RNG state and support inference or fine-tuning rather than exact
-training continuation.
-
-## Evaluation
-
-Greedy evaluation uses 15 preference vectors and independent order scenarios.
-The primary configuration has three label lines and an inline processing time
-of 0.45 s/pack. Relative to the capacity-consolidating reference set C
-(32 controllers), the policy improves three-objective hypervolume by **+0.058**
-(95% crossed-bootstrap CI **[+0.025, +0.095]**, approximately **7%**).
-An additional independent test set confirms an advantage of +0.050
-(95% CI [+0.015, +0.084]). These intervals account for training-model and
-order-scenario variability.
-
-The advantage increases with a slower inline line (0.60 s/pack). Results with
-one label line are inconclusive; at the fastest inline setting (0.35 s/pack),
-one test shows no significant difference and another shows a small disadvantage.
-The paper also compares against an extended, 43-controller family.
-
-## Files
-
-| Path | Contents |
-|---|---|
-| `ppo/seed_{1..5}/policy_200.pt` | Five final PPO state dictionaries |
-| `ppo/seed_{1..5}/metrics.csv` | Historical training metrics |
-| `bc/bc_seq_morl.pt` | Shared BC warm-start weights |
-| `inference/seq_morl/` | Minimal PyTorch architecture and configuration source |
-| `configs/paper.json` | Current paper reproduction settings |
-| `normalisation/` | Reward normalisers and observation-scale measurements |
-| `manifest.json` | Source revision, checkpoint origins and SHA-256 file hashes |
-
-The earlier PPO weights and REINFORCE ablation remain available in repository
-history; this revision contains the current paper models.
-
-## Loading a policy
-
-Install `torch` and `huggingface_hub`. The following checks loading and tensor
-dimensions using **synthetic inputs**; real decisions require the full project's
-normalised observations in the same feature order.
+## Repository structure
+
+```
+ppo/seed_{1..5}/policy_200.pt   # final PPO checkpoints (5 independent seeds)
+ppo/seed_{1..5}/metrics.csv     # training metrics
+bc/bc_seq_morl.pt               # behavioural-cloning warm start
+configs/paper.json              # training configuration
+normalisation/                  # observation and reward normalisers
+inference/seq_morl/             # minimal code to load the policy
+manifest.json                   # source revision and SHA-256 of every file
+```
+
+## Inference
 
 ```python
-import sys
+import sys, torch
 from pathlib import Path
-import torch
 from huggingface_hub import snapshot_download
 
 release = Path(snapshot_download("fhi/smartflexpack-morl"))
@@ -116,36 +66,25 @@ sys.path.insert(0, str(release / "inference"))
 from seq_morl.policies.attention import SeqAttentionActorCritic
 
 model = SeqAttentionActorCritic()
-weights = torch.load(release / "ppo/seed_1/policy_200.pt",
-                     map_location="cpu", weights_only=True)
-model.load_state_dict(weights, strict=True)
+model.load_state_dict(torch.load(release / "ppo/seed_1/policy_200.pt",
+                                 map_location="cpu", weights_only=True))
 model.eval()
 
 with torch.no_grad():
-    context = model.encode_sequence(torch.zeros(3, 5))
-    logits, value = model.forward_single(context[0], torch.zeros(22))
-    action = logits.argmax(dim=-1).item()  # 0 = INLINE, 1 = LABEL
+    ctx = model.encode_sequence(order_feats)              # [N, 32], pending orders
+    logits, value = model.forward_single(ctx[i], extra)   # extra: 10 decision + 12 global
+    action = logits.argmax(dim=-1).item()                 # 0 = inline, 1 = label
 ```
 
-For each planning event, encode its EDF-ordered pending orders once. For each
-free decision, select that order's embedding and supply the updated ten decision
-features followed by the twelve global features. Use each seed independently;
-the paper reports results across five models rather than a voting ensemble.
+Simulation and evaluation: `experiments/evaluate_policy.py` in the project
+repository; it also needs the separate DataGenerator.
 
-Full simulation and evaluation use `experiments/evaluate_policy.py` in the
-[project repository](https://fhi-git.fraunhofer.it/smartflexpack/smartflexpack_digtwin_prototype)
-(access may be required). They additionally require the separate DataGenerator
-and variants workbook, configured through `DATAGENERATOR_SRC` and
-`DATAGENERATOR_DATA`. Industrial input data are outside this model release.
+## Training
 
-## Intended use and limitations
+Behavioural-cloning warm start, then 5 independent seeds of 200 PPO updates,
+10 parallel episodes of 600 orders, lr=3e-4 (Adam). About 3.8 h CPU per seed.
 
-Research on preference-conditioned packaging scheduling in the supplied DES.
-Performance depends on the plant, planner, observation transforms and operating
-assumptions used in training and evaluation. Evidence comes from simulation;
-physical deployment and service-level guarantees require further validation.
-Changing the preference vector supports different priorities within the learned
-range; coverage of every Pareto-optimal solution is unproven.
+Training code: `experiments/train_bc_warm_start.py`, `experiments/train_policy.py`
 
 ## Citation
 
@@ -162,5 +101,6 @@ range; coverage of every Pareto-optimal solution is unproven.
 
 ## License
 
-Model weights retain the repository's [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)
-license. Fraunhofer Italia Research, EFRE1054 SMARTFLEXPACK project.
+[CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)
+Fraunhofer Italia Research, EFRE1054 Smartflexpack project
+(funded under FESR 2021–27, Priority 1 "Smart" R&I programme).

manifest.json

@@ -34,7 +34,7 @@
     "inference/seq_morl/common/layout.py": "experiments/seq_morl/common/layout.py"
   },
   "sha256": {
-    "README.md": "be0985b487a2e513e51811d5cefd7d6ab798f1551782cb0337bb2f968be34e00",
+    "README.md": "88b8100be0a8ca3b67c419be4cbd0b713dca8b20c3617259de86fa2d0330f54b",
     "bc/bc_seq_morl.pt": "cf488cf1bcedbf54f9a995e396b915650bec658a9a05b0fe791a9522416993bb",
     "configs/paper.json": "d548976847f396a057c47b49cdb5440c9800bace63d29c0d501c74e497c739ba",
     "inference/seq_morl/__init__.py": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",