Compare commits

..

11 Commits

Author SHA1 Message Date
Florent Beauchamp
17a697e750 fix(backups): better detection of looped vhd chain 2024-02-09 16:36:28 +00:00
Julien Fontanet
a2c36c0832 feat(xo-server): add robots.txt
Fixes zammad#21489
2024-02-09 11:25:06 +01:00
Florent BEAUCHAMP
2eb49cfdf1 feat: release 5.91.2 (#7367) 2024-02-09 11:10:59 +01:00
Florent BEAUCHAMP
ba9d4d4bb5 feat: technical release (#7365) 2024-02-09 10:09:09 +01:00
b-Nollet
18dea2f2fe fix(xo-server-load-balancer): create simple plan as in config (#7358)
Previously, a density plan was created when simple plan was selected in load-balancer configuration.
2024-02-08 18:14:08 +01:00
Julien Fontanet
70c51227bf feat(xo-server/rest-api): expose messages
Fixes zammad#21415
2024-02-08 11:25:05 +01:00
Julien Fontanet
e162fd835b feat(xo-server/rest-api): add /groups/:id/users and /users/:id/groups collections
Fixes https://xcp-ng.org/forum/post/70500
2024-02-08 11:23:17 +01:00
Julien Fontanet
bcdcfbf20b feat(xo-server/rest-api): add groups collection
See https://xcp-ng.org/forum/post/70500
2024-02-08 11:23:17 +01:00
Julien Fontanet
a6e93c895c chore(xo-server/rest-api): unify collections handling 2024-02-08 11:23:17 +01:00
Julien Fontanet
5c4f907358 chore(xo-server/rest-api): move :object handling to the collection 2024-02-08 09:56:04 +01:00
Julien Fontanet
e19dbc06fe chore(xo-server/rest-api): uniformize collections creation 2024-02-08 09:06:53 +01:00
12 changed files with 288 additions and 236 deletions

View File

@@ -432,6 +432,13 @@ export async function cleanVm(
if (child !== undefined) {
const chain = getUsedChildChainOrDelete(child)
if (chain !== undefined) {
if (chain.includes(vhd)) {
logWarn('loop vhd chain', { path: vhd })
// keep the current chain
// note that a VHD can't have two children, that means that
// a looped one is always the last of a chain
return chain
}
chain.unshift(vhd)
return chain
}

View File

@@ -1,5 +1,28 @@
# ChangeLog
## **5.91.2** (2024-02-09)
<img id="latest" src="https://badgen.net/badge/channel/latest/yellow" alt="Channel: latest" />
### Enhancements
- [REST API] Add `/groups` collection [Forum#70500](https://xcp-ng.org/forum/post/70500)
- [REST API] Add `/groups/:id/users` and `/users/:id/groups` collection [Forum#70500](https://xcp-ng.org/forum/post/70500)
- [REST API] Expose messages associated to XAPI objects at `/:collection/:object/messages`
### Bug fixes
- [Import/VMWare] Fix `(Failure \"Expected string, got 'I(0)'\")` (PR [#7361](https://github.com/vatesfr/xen-orchestra/issues/7361))
- [Plugin/load-balancer] Fixing `TypeError: Cannot read properties of undefined (reading 'high')` happening when trying to optimize a host with performance plan [#7359](https://github.com/vatesfr/xen-orchestra/issues/7359) (PR [#7362](https://github.com/vatesfr/xen-orchestra/pull/7362))
- Changing the number of displayed items per page should send back to the first page [#7350](https://github.com/vatesfr/xen-orchestra/issues/7350)
- [Plugin/load-balancer] Correctly create a _simple_ instead of a _density_ plan when it is selected (PR [#7358](https://github.com/vatesfr/xen-orchestra/pull/7358))
### Released packages
- xo-server 5.136.0
- xo-server-load-balancer 0.8.1
- xo-web 5.136.1
## **5.91.1** (2024-02-06)
### Bug fixes
@@ -18,8 +41,6 @@
## **5.91.0** (2024-01-31)
<img id="latest" src="https://badgen.net/badge/channel/latest/yellow" alt="Channel: latest" />
### Highlights
- [Import/VMWare] Speed up import and make all imports thin [#7323](https://github.com/vatesfr/xen-orchestra/issues/7323)

View File

@@ -7,14 +7,12 @@
> Users must be able to say: “Nice enhancement, I'm eager to test it”
- Disable search engine indexing via a `robots.txt`
### Bug fixes
> Users must be able to say: “I had this issue, happy to know it's fixed”
- [Import/VMWare] Fix `(Failure \"Expected string, got 'I(0)'\")` (PR [#7361](https://github.com/vatesfr/xen-orchestra/issues/7361))
- [Plugin/load-balancer] Fixing `TypeError: Cannot read properties of undefined (reading 'high')` happening when trying to optimize a host with performance plan [#7359](https://github.com/vatesfr/xen-orchestra/issues/7359) (PR [#7362](https://github.com/vatesfr/xen-orchestra/pull/7362))
- Changing the number of displayed items per page should send back to the first page [#7350](https://github.com/vatesfr/xen-orchestra/issues/7350)
### Packages to release
> When modifying a package, add it here with its release type.
@@ -31,8 +29,7 @@
<!--packages-start-->
- @xen-orchestra/backups patch
- xo-server patch
- xo-server-load-balancer patch
- xo-web patch
<!--packages-end-->

View File

@@ -34,9 +34,8 @@ But it's not the only way to see this: there is multiple possibilities to "optim
- maybe you want to spread the VM load on the maximum number of server, to get the most of your hardware? (previous example)
- maybe you want to reduce power consumption and migrate your VMs to the minimum number of hosts possible? (and shutdown useless hosts)
- or maybe both, depending of your own schedule?
Those ways can be also called modes: "performance" for 1, "density" for number 2 and "mixed" for the last.
Those ways can be also called modes: "performance" for 1 and "density" for number 2.
## Configure a plan
@@ -47,7 +46,6 @@ A plan has:
- a name
- pool(s) where to apply the policy
- a mode (see paragraph below)
- a behavior (aggressive, normal, low)
### Plan modes
@@ -55,7 +53,7 @@ There are 3 modes possible:
- performance
- density
- mixed
- simple
#### Performance
@@ -65,14 +63,9 @@ VMs are placed to use all possible resources. This means balance the load to giv
This time, the objective is to use the least hosts possible, and to concentrate your VMs. In this mode, you can choose to shutdown unused (and compatible) hosts.
#### Mixed
#### Simple
This mode allows you to use both performance and density, but alternatively, depending of a schedule. E.g:
- **performance** from 6:00 AM to 7:00 PM
- **density** from 7:01 PM to 5:59 AM
In this case, you'll have the best of both when needed (energy saving during the night and performance during the day).
This mode allows you to use VM anti-affinity without using any load balancing mechanism. (see paragraph below)
### Threshold
@@ -87,6 +80,10 @@ If the CPU threshold is set to 90%, the load balancer will be only triggered if
For free memory, it will be triggered if there is **less** free RAM than the threshold.
### Exclusion
If you want to prevent load balancing from triggering migrations on a particular host or VM, it is possible to exclude it from load balancing. It can be configured via the "Excluded hosts" parameter in each plan, and in the "Ignored VM tags" parameter which is common to every plan.
### Timing
The global situation (resource usage) is examined **every minute**.

View File

@@ -1,6 +1,6 @@
{
"name": "xo-server-load-balancer",
"version": "0.8.0",
"version": "0.8.1",
"license": "AGPL-3.0-or-later",
"description": "Load balancer for XO-Server",
"keywords": [

View File

@@ -12,6 +12,8 @@ import { EXECUTION_DELAY, debug } from './utils'
const PERFORMANCE_MODE = 0
const DENSITY_MODE = 1
const SIMPLE_MODE = 2
const MODES = { 'Performance mode': PERFORMANCE_MODE, 'Density mode': DENSITY_MODE, 'Simple mode': SIMPLE_MODE }
// ===================================================================
@@ -35,7 +37,7 @@ export const configurationSchema = {
},
mode: {
enum: ['Performance mode', 'Density mode', 'Simple mode'],
enum: Object.keys(MODES),
title: 'Mode',
},
@@ -147,7 +149,7 @@ class LoadBalancerPlugin {
if (plans) {
for (const plan of plans) {
this._addPlan(plan.mode === 'Performance mode' ? PERFORMANCE_MODE : DENSITY_MODE, plan)
this._addPlan(MODES[plan.mode], plan)
}
}
}

View File

@@ -143,6 +143,7 @@ port = 80
requestTimeout = 0
[http.mounts]
'/robots.txt' = './robots.txt'
'/' = '../xo-web/dist/'
'/v6' = '../../@xen-orchestra/web/dist/'

View File

@@ -1,7 +1,7 @@
{
"private": true,
"name": "xo-server",
"version": "5.135.1",
"version": "5.136.0",
"license": "AGPL-3.0-or-later",
"description": "Server part of Xen-Orchestra",
"keywords": [

View File

@@ -0,0 +1,2 @@
User-agent: *
Disallow: /

View File

@@ -100,6 +100,17 @@ async function sendObjects(iterable, req, res, path = req.path) {
return pipeline(makeObjectsStream(iterable, makeResult, json, res), res)
}
function handleArray(array, filter, limit) {
if (filter !== undefined) {
array = array.filter(filter)
}
if (limit < array.length) {
array.length = limit
}
return array
}
const handleOptionalUserFilter = filter => filter && CM.parse(filter).createPredicate()
const subRouter = (app, path) => {
@@ -160,77 +171,7 @@ export default class RestApi {
)
})
const types = [
'host',
'network',
'pool',
'SR',
'VBD',
'VDI-snapshot',
'VDI',
'VIF',
'VM-snapshot',
'VM-template',
'VM',
]
const collections = Object.fromEntries(
types.map(type => {
const id = type.toLocaleLowerCase() + 's'
return [id, { id, isCorrectType: _ => _.type === type, type }]
})
)
collections.backup = { id: 'backup' }
collections.restore = { id: 'restore' }
collections.tasks = { id: 'tasks' }
collections.users = { id: 'users' }
collections.hosts.routes = {
__proto__: null,
async 'audit.txt'(req, res) {
const host = req.xapiObject
res.setHeader('content-type', 'text/plain')
await pipeline(await host.$xapi.getResource('/audit_log', { host }), compressMaybe(req, res))
},
async 'logs.tar'(req, res) {
const host = req.xapiObject
res.setHeader('content-type', 'application/x-tar')
await pipeline(await host.$xapi.getResource('/host_logs_download', { host }), compressMaybe(req, res))
},
async missing_patches(req, res) {
await app.checkFeatureAuthorization('LIST_MISSING_PATCHES')
const host = req.xapiObject
res.json(await host.$xapi.listMissingPatches(host))
},
}
collections.pools.routes = {
__proto__: null,
async missing_patches(req, res) {
await app.checkFeatureAuthorization('LIST_MISSING_PATCHES')
const xapi = req.xapiObject.$xapi
const missingPatches = new Map()
await asyncEach(Object.values(xapi.objects.indexes.type.host ?? {}), async host => {
try {
for (const patch of await xapi.listMissingPatches(host)) {
const { uuid: key = `${patch.name}-${patch.version}-${patch.release}` } = patch
missingPatches.set(key, patch)
}
} catch (error) {
console.warn(host.uuid, error)
}
})
res.json(Array.from(missingPatches.values()))
},
}
const collections = { __proto__: null }
const withParams = (fn, paramsSchema) => {
fn.params = paramsSchema
@@ -238,68 +179,231 @@ export default class RestApi {
return fn
}
collections.pools.actions = {
__proto__: null,
{
const types = [
'host',
'message',
'network',
'pool',
'SR',
'VBD',
'VDI-snapshot',
'VDI',
'VIF',
'VM-snapshot',
'VM-template',
'VM',
]
function getObject(id, req) {
const { type } = this
const object = app.getObject(id, type)
create_vm: withParams(
defer(async ($defer, { xapiObject: { $xapi } }, { affinity, boot, install, template, ...params }, req) => {
params.affinityHost = affinity
params.installRepository = install?.repository
// add also the XAPI version of the object
req.xapiObject = app.getXapiObject(object)
const vm = await $xapi.createVm(template, params, undefined, req.user.id)
$defer.onFailure.call($xapi, 'VM_destroy', vm.$ref)
return object
}
function getObjects(filter, limit) {
return app.getObjects({
filter: every(this.isCorrectType, filter),
limit,
})
}
async function messages(req, res) {
const {
object: { id },
query,
} = req
await sendObjects(
app.getObjects({
filter: every(_ => _.type === 'message' && _.$object === id, handleOptionalUserFilter(query.filter)),
limit: ifDef(query.limit, Number),
}),
req,
res,
'/messages'
)
}
for (const type of types) {
const id = type.toLocaleLowerCase() + 's'
if (boot) {
await $xapi.callAsync('VM.start', vm.$ref, false, false)
}
collections[id] = { getObject, getObjects, routes: { messages }, isCorrectType: _ => _.type === type, type }
}
return vm.uuid
}),
{
affinity: { type: 'string', optional: true },
auto_poweron: { type: 'boolean', optional: true },
boot: { type: 'boolean', default: false },
clone: { type: 'boolean', default: true },
install: {
type: 'object',
optional: true,
properties: {
method: { enum: ['cdrom', 'network'] },
repository: { type: 'string' },
collections.hosts.routes = {
...collections.hosts.routes,
async 'audit.txt'(req, res) {
const host = req.xapiObject
res.setHeader('content-type', 'text/plain')
await pipeline(await host.$xapi.getResource('/audit_log', { host }), compressMaybe(req, res))
},
async 'logs.tar'(req, res) {
const host = req.xapiObject
res.setHeader('content-type', 'application/x-tar')
await pipeline(await host.$xapi.getResource('/host_logs_download', { host }), compressMaybe(req, res))
},
async missing_patches(req, res) {
await app.checkFeatureAuthorization('LIST_MISSING_PATCHES')
const host = req.xapiObject
res.json(await host.$xapi.listMissingPatches(host))
},
}
collections.pools.routes = {
...collections.pools.routes,
async missing_patches(req, res) {
await app.checkFeatureAuthorization('LIST_MISSING_PATCHES')
const xapi = req.xapiObject.$xapi
const missingPatches = new Map()
await asyncEach(Object.values(xapi.objects.indexes.type.host ?? {}), async host => {
try {
for (const patch of await xapi.listMissingPatches(host)) {
const { uuid: key = `${patch.name}-${patch.version}-${patch.release}` } = patch
missingPatches.set(key, patch)
}
} catch (error) {
console.warn(host.uuid, error)
}
})
res.json(Array.from(missingPatches.values()))
},
}
collections.pools.actions = {
create_vm: withParams(
defer(async ($defer, { xapiObject: { $xapi } }, { affinity, boot, install, template, ...params }, req) => {
params.affinityHost = affinity
params.installRepository = install?.repository
const vm = await $xapi.createVm(template, params, undefined, req.user.id)
$defer.onFailure.call($xapi, 'VM_destroy', vm.$ref)
if (boot) {
await $xapi.callAsync('VM.start', vm.$ref, false, false)
}
return vm.uuid
}),
{
affinity: { type: 'string', optional: true },
auto_poweron: { type: 'boolean', optional: true },
boot: { type: 'boolean', default: false },
clone: { type: 'boolean', default: true },
install: {
type: 'object',
optional: true,
properties: {
method: { enum: ['cdrom', 'network'] },
repository: { type: 'string' },
},
},
memory: { type: 'integer', optional: true },
name_description: { type: 'string', minLength: 0, optional: true },
name_label: { type: 'string' },
template: { type: 'string' },
}
),
emergency_shutdown: async ({ xapiObject }) => {
await app.checkFeatureAuthorization('POOL_EMERGENCY_SHUTDOWN')
await xapiObject.$xapi.pool_emergencyShutdown()
},
rolling_update: async ({ object }) => {
await app.checkFeatureAuthorization('ROLLING_POOL_UPDATE')
await app.rollingPoolUpdate(object)
},
}
collections.vms.actions = {
clean_reboot: ({ xapiObject: vm }) => vm.$callAsync('clean_reboot').then(noop),
clean_shutdown: ({ xapiObject: vm }) => vm.$callAsync('clean_shutdown').then(noop),
hard_reboot: ({ xapiObject: vm }) => vm.$callAsync('hard_reboot').then(noop),
hard_shutdown: ({ xapiObject: vm }) => vm.$callAsync('hard_shutdown').then(noop),
snapshot: withParams(
async ({ xapiObject: vm }, { name_label }) => {
const ref = await vm.$snapshot({ name_label })
return vm.$xapi.getField('VM', ref, 'uuid')
},
memory: { type: 'integer', optional: true },
name_description: { type: 'string', minLength: 0, optional: true },
name_label: { type: 'string' },
template: { type: 'string' },
}
),
emergency_shutdown: async ({ xapiObject }) => {
await app.checkFeatureAuthorization('POOL_EMERGENCY_SHUTDOWN')
{ name_label: { type: 'string', optional: true } }
),
start: ({ xapiObject: vm }) => vm.$callAsync('start', false, false).then(noop),
}
}
await xapiObject.$xapi.pool_emergencyShutdown()
collections.backup = {}
collections.groups = {
getObject(id) {
return app.getGroup(id)
},
rolling_update: async ({ xoObject }) => {
await app.checkFeatureAuthorization('ROLLING_POOL_UPDATE')
await app.rollingPoolUpdate(xoObject)
async getObjects(filter, limit) {
return handleArray(await app.getAllGroups(), filter, limit)
},
routes: {
async users(req, res) {
const { filter, limit } = req.query
await sendObjects(
handleArray(
await Promise.all(req.object.users.map(id => app.getUser(id).then(getUserPublicProperties))),
handleOptionalUserFilter(filter),
ifDef(limit, Number)
),
req,
res,
'/users'
)
},
},
}
collections.vms.actions = {
__proto__: null,
clean_reboot: ({ xapiObject: vm }) => vm.$callAsync('clean_reboot').then(noop),
clean_shutdown: ({ xapiObject: vm }) => vm.$callAsync('clean_shutdown').then(noop),
hard_reboot: ({ xapiObject: vm }) => vm.$callAsync('hard_reboot').then(noop),
hard_shutdown: ({ xapiObject: vm }) => vm.$callAsync('hard_shutdown').then(noop),
snapshot: withParams(
async ({ xapiObject: vm }, { name_label }) => {
const ref = await vm.$snapshot({ name_label })
return vm.$xapi.getField('VM', ref, 'uuid')
collections.restore = {}
collections.tasks = {}
collections.users = {
getObject(id) {
return app.getUser(id).then(getUserPublicProperties)
},
async getObjects(filter, limit) {
return handleArray(await app.getAllUsers(), filter, limit)
},
routes: {
async groups(req, res) {
const { filter, limit } = req.query
await sendObjects(
handleArray(
await Promise.all(req.object.groups.map(id => app.getGroup(id))),
handleOptionalUserFilter(filter),
ifDef(limit, Number)
),
req,
res,
'/groups'
)
},
{ name_label: { type: 'string', optional: true } }
),
start: ({ xapiObject: vm }) => vm.$callAsync('start', false, false).then(noop),
},
}
// normalize collections
for (const id of Object.keys(collections)) {
const collection = collections[id]
// inject id into the collection
collection.id = id
// set null as prototypes to speed-up look-ups
Object.setPrototypeOf(collection, null)
const { actions, routes } = collection
if (actions !== undefined) {
Object.setPrototypeOf(actions, null)
}
if (routes !== undefined) {
Object.setPrototypeOf(routes, null)
}
}
api.param('collection', (req, res, next) => {
@@ -312,14 +416,14 @@ export default class RestApi {
next()
}
})
api.param('object', (req, res, next) => {
api.param('object', async (req, res, next) => {
const id = req.params.object
const { type } = req.collection
try {
req.xapiObject = app.getXapiObject((req.xoObject = app.getObject(id, type)))
next()
// eslint-disable-next-line require-atomic-updates
req.object = await req.collection.getObject(id, req)
return next()
} catch (error) {
if (noSuchObject.is(error, { id, type })) {
if (noSuchObject.is(error, { id })) {
next('route')
} else {
next(error)
@@ -478,39 +582,12 @@ export default class RestApi {
}, true)
)
api
.get(
'/users',
wrap(async (req, res) => {
let users = await app.getAllUsers()
const { filter, limit } = req.query
if (filter !== undefined) {
users = users.filter(CM.parse(filter).createPredicate())
}
if (limit < users.length) {
users.length = limit
}
sendObjects(users.map(getUserPublicProperties), req, res)
})
)
.get(
'/users/:id',
wrap(async (req, res) => {
res.json(getUserPublicProperties(await app.getUser(req.params.id)))
})
)
api.get(
'/:collection',
wrap(async (req, res) => {
const { query } = req
await sendObjects(
await app.getObjects({
filter: every(req.collection.isCorrectType, handleOptionalUserFilter(query.filter)),
limit: ifDef(query.limit, Number),
}),
await req.collection.getObjects(handleOptionalUserFilter(query.filter), ifDef(query.limit, Number)),
req,
res
)
@@ -563,7 +640,7 @@ export default class RestApi {
)
api.get('/:collection/:object', (req, res) => {
let result = req.xoObject
let result = req.object
// add locations of sub-routes for discoverability
const { routes } = req.collection
@@ -618,7 +695,7 @@ export default class RestApi {
'/:collection/:object/tasks',
wrap(async (req, res) => {
const { query } = req
const objectId = req.xoObject.id
const objectId = req.object.id
const tasks = app.tasks.list({
filter: every(
_ => _.status === 'pending' && _.properties.objectId === objectId,
@@ -658,9 +735,9 @@ export default class RestApi {
}
}
const { xapiObject, xoObject } = req
const task = app.tasks.create({ name: `REST: ${action} ${req.collection.type}`, objectId: xoObject.id })
const pResult = task.run(() => fn({ xapiObject, xoObject }, params, req))
const { object, xapiObject } = req
const task = app.tasks.create({ name: `REST: ${action} ${req.collection.type}`, objectId: object.id })
const pResult = task.run(() => fn({ object, xapiObject }, params, req))
if (Object.hasOwn(req.query, 'sync')) {
pResult.then(result => res.json(result), next)
} else {

View File

@@ -1,7 +1,7 @@
{
"private": true,
"name": "xo-web",
"version": "5.136.0",
"version": "5.136.1",
"license": "AGPL-3.0-or-later",
"description": "Web interface client for Xen-Orchestra",
"keywords": [

View File

@@ -1,52 +0,0 @@
#!/usr/bin/env node
import https from 'node:https'
import tls from 'node:tls'
const [, , host, port = 443] = process.argv
async function tryRequest(options) {
https
.request(options, res => {
console.log('statusCode:', res.statusCode)
})
.on('error', function (error) {
console.error('error:', error)
})
.end()
}
function getCertificate(options) {
return new Promise((resolve, reject) => {
tls
.connect(options, function () {
resolve(this.getPeerX509Certificate())
this.end()
})
.on('error', function (error) {
this.destroy()
reject(error)
})
})
}
// Trying request : it fails if self-signed certificate
const options = { host, port, rejectUnauthorized: true, servername: host }
console.log('\n-> Request with no certificate')
tryRequest(options)
// Asking for certificate
const certificate = await getCertificate({ ...options, rejectUnauthorized: false })
console.log('=> Certificate acquired')
// console.log(certificate.subject, certificate.issuer, certificate.validFrom, certificate.validTo)
// console.log(certificate.verify(certificate.publicKey))
// Trying again with certificate
console.log('\n-> Request using acquired certificate')
tryRequest({
...options,
ca: [...tls.rootCertificates, certificate.toString()],
// adding default ca with ...tls.rootCertificates avoids failing requests with other valid certificates, but it looks likes it also makes succeeding requests we want to fail (like https://pinning-test.badssl.com/)
// checkServerIdentity: () => {return undefined}, // for localhost
})