Adds a jobs queue to Payload.
- [x] Docs, w/ examples for Vercel Cron, additional services
- [x] Type the `job` using GeneratedTypes in `JobRunnerArgs`
(@AlessioGr)
- [x] Write the `runJobs` function
- [x] Allow for some type of `payload.runTask`
- [x] Open up a new bin script for running jobs
- [x] Determine strategy for runner endpoint to either await jobs
successfully or return early and stay open until job work completes
(serverless ramifications here)
- [x] Allow for job runner to accept how many jobs to run in one
invocation
- [x] Make a Payload local API method for creating a new job easily
(payload.createJob) or similar which is strongly typed (@AlessioGr)
- [x] Make `payload.runJobs` or similar (@AlessioGr)
- [x] Write tests for retrying up to max retries for a given step
- [x] Write tests for dynamic import of a runner
The shape of the config should permit the definition of steps separate
from the job workflows themselves.
```js
const config = {
// Not sure if we need this property anymore
queues: {
},
// A job is an instance of a workflow, stored in DB
// and triggered by something at some point
jobs: {
// Be able to override the jobs collection
collectionOverrides: () => {},
// Workflows are groups of tasks that handle
// the flow from task to task.
// When defined on the config, they are considered as predefined workflows
// BUT - in the future, we'll allow for UI-based workflow definition as well.
workflows: [
{
slug: 'job-name',
// Temporary name for this
// should be able to pass function
// or path to it for Node to dynamically import
controlFlowInJS: '/my-runner.js',
// Temporary name as well
// should be able to eventually define workflows
// in UI (meaning they need to be serialized in JSON)
// Should not be able to define both control flows
controlFlowInJSON: [
{
task: 'myTask',
next: {
// etc
}
}
],
// Workflows take input
// which are a group of fields
input: [
{
name: 'post',
type: 'relationship',
relationTo: 'posts',
maxDepth: 0,
required: true,
},
{
name: 'message',
type: 'text',
required: true,
},
],
},
],
// Tasks are defined separately as isolated functions
// that can be retried on fail
tasks: [
{
slug: 'myTask',
retries: 2,
// Each task takes input
// Used to auto-type the task func args
input: [
{
name: 'post',
type: 'relationship',
relationTo: 'posts',
maxDepth: 0,
required: true,
},
{
name: 'message',
type: 'text',
required: true,
},
],
// Each task takes output
// Used to auto-type the function signature
output: [
{
name: 'success',
type: 'checkbox',
}
],
onSuccess: () => {},
onFail: () => {},
run: myRunner,
},
]
}
}
```
### `payload.createJob`
This function should allow for the creation of jobs based on either a
workflow (group of tasks) or an individual task.
To create a job using a workflow:
```js
const job = await payload.createJob({
// Accept the `name` of a workflow so we can match to either a
// code-based workflow OR a workflow defined in the DB
// Should auto-type the input
workflowName: 'myWorkflow',
input: {
// typed to the args of the workflow by name
}
})
```
To create a job using a task:
```js
const job = await payload.createJob({
// Accept the `name` of a task
task: 'myTask',
input: {
// typed to the args of the task by name
}
})
```
---------
Co-authored-by: Alessio Gravili <alessio@gravili.de>
Co-authored-by: Dan Ribbens <dan.ribbens@gmail.com>
Fixes https://github.com/payloadcms/payload/issues/8013
**BREAKING:**
- Upgrades minimum supported @types/react version from
npm:types-react@19.0.0-rc.0 to npm:types-react@19.0.0-rc.1
- Upgrades minimum supported @types/react-dom version from
npm:types-react-dom@19.0.0-rc.0 to npm:types-react-dom@19.0.0-rc.1
- Upgrades minimum supported react and react-dom version from
19.0.0-rc-06d0b89e-20240801 to 19.0.0-rc-5dcb0097-20240918
- Upgrades minimum supported Next.js version from 15.0.0-canary.104 to
15.0.0-canary.160
---------
Co-authored-by: PatrikKozak <patrik@payloadcms.com>
Co-authored-by: Jacob Fletcher <jacobsfletch@gmail.com>
Fixes https://github.com/payloadcms/payload/issues/8111 and
https://github.com/payloadcms/payload/issues/8113
Before: 132 dependencies
After: 123 dependencies
This PR also contains a small performance optimization during telemetry
startup: By using the async `fs.promises.readFile` instead of
`readFileSync` we're not blocking the entire thread anymore and are
allowing other stuff to happen while the file is being read.
Also, in our dependency checker, this moves some variables out of loops,
to the module scope, as they only need to be calculated once.
We have to pin file-type to 19.3.0 and cannot upgrade it further (latest
is 19.5.0). See reasoning in
https://github.com/payloadcms/payload/issues/8111#issuecomment-2348119533
Ajv 8.14.0 => 8.17.1
- Bundle size: 119.6kB => 111kB
- Dependencies: 5 => 4
- Gets rid of dependency on `punycode`. Will help with the annoying
deprecated module console warning spam
This also upgrades TypeScript to 5.6.2 in our monorepo. The most
type-relevant packages are updated as well, e.g. ts-essentials and
@types/node
Elliot DeNolf