# Mutual Fund — NAV / profile scrape (frontend integration)

Use this file to add **Update all**, **Refresh NAV**, **Refresh profiles**, a **run list**, and an optional **Rebuild K(t)** button.

**Base URL:** `{APP_URL}/api`  
**Example:** `http://127.0.0.1:8000/api`  
**Controller:** `MfScrapeController`  
**Auth:** these MF routes currently use only the `api` middleware (no JWT). Send `Accept: application/json`. Pass `users_id` when a back-office user is logged in.

Do **not** wait on the POST. It only queues work. Poll the batch or the run list until `finished` is true.

---

## Response envelope

```json
{
  "status": true,
  "message": "optional text",
  "data": {},
  "pagination": {},
  "errors": {},
  "error": "server error text"
}
```

| HTTP | When |
|------|------|
| 200 | Queued, list, show, batch, retry, rebuild |
| 404 | Empty run list, unknown run/batch, no matching funds |
| 422 | Validation failed — read `errors.{field}[]` |
| 500 | Unexpected server error |

Treat `status === false` as failure even if HTTP is 200.

---

## Screens and endpoints

| Screen / control | Method | Path |
|------------------|--------|------|
| Update all (NAV + profiles) | POST | `/mf-scrape/mf-scrape-all` |
| Refresh NAV | POST | `/mf-scrape/mf-scrape-nav` |
| Refresh profiles + dividends | POST | `/mf-scrape/mf-scrape-profiles` |
| Rebuild K(t) without scrape | POST | `/mf-scrape/mf-scrape-rebuild-returns` |
| Run list | GET | `/mf-scrape/mf-scrape-index` |
| Run detail | GET | `/mf-scrape/mf-scrape-show/{id}` |
| Batch progress | GET | `/mf-scrape/mf-scrape-batch/{batchId}` |
| Retry failed run | POST | `/mf-scrape/mf-scrape-retry/{id}` |

---

## Suggested UI

1. **Update all funds** button → `POST /mf-scrape/mf-scrape-all` with `{ "scope": "all", "include_profiles": true, "users_id": <id> }`.
2. Show toast: `Queued {nav_queued} NAV runs`. Save `data.batch_id`.
3. Poll `GET /mf-scrape/mf-scrape-batch/{batchId}` every 3–5s until `data.finished === true`.
4. Optional second poll on `profile_batch_id` when profiles were included.
5. Run list table: type, fund, status, records imported, duration, message, Retry on `failed`.

Optional filters on the buttons: `scope=stale`, `scope=selected` + `fund_ids`, or `amc_id`.

Rebuild K(t) is the same math as **Return Series → Sync adjusted NAV**. Use it only when an admin wants to recalculate without hitting AMC sites.

---

## Queue POST body

| Field | Type | Default | UI |
|-------|------|---------|----|
| `scope` | `all` / `stale` / `selected` | `all` | radio |
| `fund_ids` | int[] | — | required if selected |
| `amc_id` | int | — | AMC filter |
| `include_profiles` | bool | true | scrape-all checkbox |
| `users_id` | int | — | logged-in back-office user |

`stale` = `latest_nav_date` is null or older than 2 days.

---

## Run list

`GET /mf-scrape/mf-scrape-index?per_page=10&status=failed`

| Query | Type |
|-------|------|
| `per_page` | 1–100 |
| `fund_id` | int |
| `amc_id` | int |
| `type` | `nav` / `fund_profile` |
| `status` | `queued` / `running` / `success` / `failed` |
| `batch_id` | uuid |

Each row includes `fund.fund_name`, `amc.slug`, counts, timestamps (`created_at` / `started_at` / `finished_at`).

---

## Worker reminder

The UI cannot finish a scrape by itself. Ops must run:

```bash
php artisan queue:work --timeout=600
```

If the list stays on `queued`, the worker is down — show that in the empty/running state, do not re-POST the same bulk action in a loop (duplicate runs are skipped for 30 minutes, but the user should not hammer the button).
