ui: show project instance volumes in default view

[1fanwang]

Description

Opening a project instance from Default view shows an empty Volumes tab even though its disks exist, because the volume list skips project resources unless the request names a project. The tab now sends the instance's own project, so the disks appear without switching projects, for root admins and domain admins alike. An instance that belongs to no project sends none, so its request is unchanged.

The tab does not ask the server for every project at once. That request reaches only the caller's own account and the projects they belong to, so a domain admin would see an empty tab for other users' instances and for projects they have not joined.

Fixes: #14070

Types of changes

• Breaking change (fix or feature that would cause existing functionality to change)
• New feature (non-breaking change which adds functionality)
• Bug fix (non-breaking change which fixes an issue)
• Enhancement (improves an existing feature and functionality)
• Cleanup (Code refactoring and cleanup, that may add test cases)
• Build/CI
• Test (unit or integration test code)

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

• Major
• Minor

Bug Severity

• BLOCKER
• Critical
• Major
• Minor
• Trivial

Screenshots (if appropriate):

The reproduction checks the rendered table rows; the layout is unchanged.

How Has This Been Tested?

A unit test in this diff covers the same cases against a local HTTP fixture that follows the server's listing rules. The run below uses a real server.

E2E

The real Volumes tab ran against CloudStack 4.22.1.0 with MySQL 8.0.46 and the Simulator hypervisor, logged in as each role. Marvin deployed the standard simulator datacenter. The fixtures come from API calls with no database edits: a domain with a domain admin and a user, a project the domain admin belongs to, a project owned by the user, an instance in each, an account-owned instance, and the secondary storage system VM. The test adds only the login session a browser would hold and selects the project view through browser local storage. The request, the response and the component are real.

#	Scenario	Volumes shown at 9030443 (base)	Volumes shown at 7e0ef8e (this change)	Expected
1	Project instance, Default view, domain admin who belongs to the project	0	2	2
2	Project instance, Default view, domain admin who does not belong	0	2	2
3	Project instance, Default view, root admin	0	2	2
4	Project instance, another project selected, domain admin	0	2	2
5	Project instance, its own project selected, domain admin	2	2	2
6	Account instance, Default view, domain admin	2	2	2
7	Account instance, Default view, root admin	2	2	2
8	Secondary storage system VM, Default view, root admin	1	1	1

Save the three reproducer files below (create_fixtures.py, probe_server.py and volumes.spec.js) in one directory and run the commands from it. The UI commands use a CloudStack checkout with the ui dependencies installed. Start the simulator and create the fixtures:

export E2E_DIR="$PWD"
export CLOUDSTACK_CHECKOUT=/path/to/cloudstack
docker run -d --rm --platform linux/amd64 --name cloudstack-volumes-14158 \
  -p 127.0.0.1:18070:5050 -p 127.0.0.1:18071:8096 -p 127.0.0.1:18072:8080 \
  apache/cloudstack-simulator@sha256:32005782746a39f3d3267ddb68a535c0e22c7ef31f145c18930da355514b1cc5
curl --silent 'http://127.0.0.1:18071/client/api?command=listZones&response=json'
docker exec cloudstack-volumes-14158 python3 /root/tools/marvin/marvin/deployDataCenter.py \
  -i /root/setup/dev/advanced.cfg -l /tmp/volumes-repro-datacenter
python3 create_fixtures.py > fixtures.json

What the server returns for each caller, instance and projectid value. A domain admin gets nothing for another user's instance, or for a project they have not joined, when the request asks for all projects (projectid=-1), and gets the volumes when it names the instance's own project:

python3 probe_server.py fixtures.json

caller        instance                               projectid sent  volumes
domain admin  account VM                             (none)          2
domain admin  account VM                             -1              0
domain admin  project VM (domain admin is a member)  (none)          0
domain admin  project VM (domain admin is a member)  -1              2
domain admin  project VM (domain admin is a member)  its own project 2
domain admin  project VM (not a member)              (none)          0
domain admin  project VM (not a member)              -1              0
domain admin  project VM (not a member)              its own project 2
root admin    account VM                             (none)          2
root admin    account VM                             -1              2
root admin    project VM (domain admin is a member)  (none)          0
root admin    project VM (domain admin is a member)  -1              2
root admin    project VM (domain admin is a member)  its own project 2
root admin    project VM (not a member)              (none)          0
root admin    project VM (not a member)              -1              2
root admin    project VM (not a member)              its own project 2

The component against the same server. For the base run the base file is put in place first. The logged projectid is undefined when the request carried none. Before, at 9030443:

export CLOUDSTACK_LIVE_FIXTURES="$(jq -c . "$E2E_DIR/fixtures.json")"
cd "$CLOUDSTACK_CHECKOUT/ui"
git show 9030443eac555c26b06bbf8e82249f2aac385fcc:ui/src/components/view/VolumesTab.vue > src/components/view/VolumesTab.vue
npm run test:unit -- --runInBand --coverage=false --verbose --roots "$E2E_DIR" --testMatch '**/volumes.spec.js' --modulePaths "$PWD/node_modules"

project VM (member), default view as domain admin: sent projectid=undefined -> rendered 0 volumes; expected 2
project VM (not a member), default view as domain admin: sent projectid=undefined -> rendered 0 volumes; expected 2
project VM (member), default view as root admin: sent projectid=undefined -> rendered 0 volumes; expected 2
project VM (member), its own project as domain admin: sent projectid=3a16f567-0544-4120-8515-3d3655ac35cb -> rendered 2 volumes; expected 2
project VM (member), another project as domain admin: sent projectid=d19b266b-20c3-448d-8bc9-5a98a4acff2e -> rendered 0 volumes; expected 2
account VM, default view as domain admin: sent projectid=undefined -> rendered 2 volumes; expected 2
account VM, default view as root admin: sent projectid=undefined -> rendered 2 volumes; expected 2
system VM, default view as root admin: sent projectid=undefined -> rendered 1 volumes; expected 1

After, at 7e0ef8e:

git checkout -- src/components/view/VolumesTab.vue
npm run test:unit -- --runInBand --coverage=false --verbose --roots "$E2E_DIR" --testMatch '**/volumes.spec.js' --modulePaths "$PWD/node_modules"

project VM (member), default view as domain admin: sent projectid=3a16f567-0544-4120-8515-3d3655ac35cb -> rendered 2 volumes; expected 2
project VM (not a member), default view as domain admin: sent projectid=13dad3c6-d72e-4585-a5dc-9da0a8c1348a -> rendered 2 volumes; expected 2
project VM (member), default view as root admin: sent projectid=3a16f567-0544-4120-8515-3d3655ac35cb -> rendered 2 volumes; expected 2
project VM (member), its own project as domain admin: sent projectid=3a16f567-0544-4120-8515-3d3655ac35cb -> rendered 2 volumes; expected 2
project VM (member), another project as domain admin: sent projectid=3a16f567-0544-4120-8515-3d3655ac35cb -> rendered 2 volumes; expected 2
account VM, default view as domain admin: sent projectid=undefined -> rendered 2 volumes; expected 2
account VM, default view as root admin: sent projectid=undefined -> rendered 2 volumes; expected 2
system VM, default view as root admin: sent projectid=undefined -> rendered 1 volumes; expected 1

Reproducer source: create_fixtures.py

import json
import sys
import time
import urllib.parse
import urllib.request

API = "http://127.0.0.1:18072/client/api"
PASSWORD = "Volrepro-1!"


def post(params: dict[str, str], cookie: str = "") -> tuple[dict, list[str]]:
    request = urllib.request.Request(
        API, data=urllib.parse.urlencode(params).encode(), method="POST", headers={"Cookie": cookie}
    )
    with urllib.request.urlopen(request, timeout=180) as response:
        return json.load(response), [value.split(";")[0] for value in response.headers.get_all("Set-Cookie") or []]


def login(username: str, password: str, domain: str) -> dict[str, str]:
    payload, cookies = post(
        {"command": "login", "username": username, "password": password, "domain": domain, "response": "json"}
    )
    return {"sessionkey": payload["loginresponse"]["sessionkey"], "cookie": "; ".join(cookies)}


ROOT = login("admin", "password", "/")


def call(command: str, **params: str) -> dict:
    return post(
        {"command": command, "response": "json", "sessionkey": ROOT["sessionkey"], **params}, ROOT["cookie"]
    )[0]


def wait(jobid: str) -> dict:
    for _ in range(180):
        result = call("queryAsyncJobResult", jobid=jobid)["queryasyncjobresultresponse"]
        if result["jobstatus"] == 1:
            return result["jobresult"]
        if result["jobstatus"] == 2:
            raise RuntimeError(json.dumps(result))
        time.sleep(2)
    raise TimeoutError(jobid)


def main() -> None:
    zone = call("listZones")["listzonesresponse"]["zone"][0]["id"]
    template = call(
        "listTemplates", templatefilter="executable", hypervisor="Simulator", zoneid=zone
    )["listtemplatesresponse"]["template"][0]["id"]
    offering = call("listServiceOfferings", issystem="false")["listserviceofferingsresponse"][
        "serviceoffering"
    ][0]["id"]
    disk = next(
        item["id"]
        for item in call("listDiskOfferings")["listdiskofferingsresponse"]["diskoffering"]
        if not item["iscustomized"] and item["disksize"] > 0
    )
    roles = call("listRoles")["listrolesresponse"]["role"]
    role = {
        kind: next(item["id"] for item in roles if item["type"] == kind and item.get("isdefault"))
        for kind in ("DomainAdmin", "User")
    }

    domain = call("createDomain", name="volrepro")["createdomainresponse"]["domain"]["id"]
    for name, kind in (("volrepro-admin", "DomainAdmin"), ("volrepro-user", "User")):
        call(
            "createAccount", username=name, password=PASSWORD, email=name + "@example.com",
            firstname="Vol", lastname="Repro", account=name, domainid=domain, roleid=role[kind],
        )
    projects = {}
    for name, owner in (
        ("volume-repro", "volrepro-admin"),
        ("other-project", "volrepro-admin"),
        ("user-project", "volrepro-user"),
    ):
        job = call(
            "createProject", name=name, displaytext=name, account=owner, domainid=domain
        )["createprojectresponse"]["jobid"]
        projects[name] = wait(job)["project"]["id"]

    def deploy(name: str, **owner: str) -> dict:
        job = call(
            "deployVirtualMachine", name=name, displayname=name, zoneid=zone, templateid=template,
            serviceofferingid=offering, diskofferingid=disk, **owner,
        )["deployvirtualmachineresponse"]["jobid"]
        return wait(job)["virtualmachine"]

    project_vm = deploy("volumes-project", projectid=projects["volume-repro"])
    user_project_vm = deploy("volumes-user-project", projectid=projects["user-project"])
    account_vm = deploy("volumes-account", account="volrepro-user", domainid=domain)

    def volumes(**params: str) -> list[str]:
        listed = call("listVolumes", listall="true", **params)["listvolumesresponse"]
        return sorted(item["id"] for item in listed.get("volume", []))

    project_volumes = volumes(virtualmachineid=project_vm["id"], projectid=projects["volume-repro"])
    user_project_volumes = volumes(virtualmachineid=user_project_vm["id"], projectid=projects["user-project"])
    account_volumes = volumes(virtualmachineid=account_vm["id"])
    assert len(project_volumes) == len(user_project_volumes) == len(account_volumes) == 2
    system_vm = call("listSystemVms")["listsystemvmsresponse"]["systemvm"][0]
    system_volumes = volumes(virtualmachineid=system_vm["id"], listsystemvms="true")
    assert system_volumes, system_vm["id"]

    print(json.dumps({
        "callers": {"domain admin": login("volrepro-admin", PASSWORD, "/volrepro"), "root admin": ROOT},
        "otherProjectId": projects["other-project"],
        "project": {
            "resource": {"id": project_vm["id"], "projectid": projects["volume-repro"]},
            "volumeIds": project_volumes,
        },
        "userProject": {
            "resource": {"id": user_project_vm["id"], "projectid": projects["user-project"]},
            "volumeIds": user_project_volumes,
        },
        "account": {"resource": {"id": account_vm["id"]}, "volumeIds": account_volumes},
        "system": {
            "resource": {"id": system_vm["id"], "systemvmtype": system_vm["systemvmtype"]},
            "volumeIds": system_volumes,
        },
    }, sort_keys=True))


if __name__ == "__main__":
    sys.exit(main())

Reproducer source: probe_server.py

from __future__ import annotations

import json
import sys
import urllib.parse
import urllib.request

API = "http://127.0.0.1:18072/client/api"
fixtures = json.load(open(sys.argv[1]))

CASES = [
    ("account VM", fixtures["account"]["resource"]["id"], [None, "-1"]),
    ("project VM (domain admin is a member)", fixtures["project"]["resource"]["id"], [None, "-1", fixtures["project"]["resource"]["projectid"]]),
    ("project VM (not a member)", fixtures["userProject"]["resource"]["id"], [None, "-1", fixtures["userProject"]["resource"]["projectid"]]),
]


def count(role: str, vm: str, projectid: str | None) -> int:
    session = fixtures["callers"][role]
    params = {"command": "listVolumes", "response": "json", "sessionkey": session["sessionkey"], "listall": "true", "listsystemvms": "true", "virtualmachineid": vm}
    if projectid:
        params["projectid"] = projectid
    request = urllib.request.Request(API, data=urllib.parse.urlencode(params).encode(), method="POST", headers={"Cookie": session["cookie"]})
    with urllib.request.urlopen(request, timeout=60) as response:
        return json.load(response)["listvolumesresponse"].get("count", 0)


print(f"{'caller':13} {'instance':38} {'projectid sent':15} volumes")
for role in ("domain admin", "root admin"):
    for name, vm, values in CASES:
        for value in values:
            label = "(none)" if value is None else ("-1" if value == "-1" else "its own project")
            print(f"{role:13} {name:38} {label:15} {count(role, vm, value)}")

Reproducer source: volumes.spec.js

import { config, flushPromises } from '@vue/test-utils'

import common from '@/../tests/common'
import VolumesTab from '@/components/view/VolumesTab.vue'
import { axios } from '@/utils/request'
import { CURRENT_PROJECT } from '@/store/mutation-types'
import { vueProps } from '@/vue-app'

jest.mock('@/vue-app', () => ({
  vueProps: {
    $localStorage: {
      get: jest.fn(() => null)
    }
  }
}))

const fixtures = JSON.parse(process.env.CLOUDSTACK_LIVE_FIXTURES)
const router = common.createMockRouter()
const i18n = common.createMockI18n('en')
let session
let wrapper
let interceptor
let sent

config.global.provide.parentFetchData = () => {}
axios.defaults.baseURL = 'http://127.0.0.1:18072/client/api'

// Only the login session a browser would hold is added; the request, the response and the component are real.
axios.interceptors.request.use(request => {
  request.params = { ...request.params, sessionkey: session.sessionkey }
  request.headers.Cookie = session.cookie
  return request
})
const httpAdapter = require('axios/lib/adapters/http')
axios.defaults.adapter = request => {
  const { sessionkey, ...params } = request.params
  sent = params
  return httpAdapter(request)
}

afterEach(() => {
  wrapper?.unmount()
  axios.interceptors.response.eject(interceptor)
})

it.each([
  ['project VM (member), default view', 'domain admin', fixtures.project, null],
  ['project VM (not a member), default view', 'domain admin', fixtures.userProject, null],
  ['project VM (member), default view', 'root admin', fixtures.project, null],
  ['project VM (member), its own project', 'domain admin', fixtures.project, fixtures.project.resource.projectid],
  ['project VM (member), another project', 'domain admin', fixtures.project, fixtures.otherProjectId],
  ['account VM, default view', 'domain admin', fixtures.account, null],
  ['account VM, default view', 'root admin', fixtures.account, null],
  ['system VM, default view', 'root admin', fixtures.system, null]
])('%s as %s shows its real volumes', async (name, role, fixture, selectedProjectId) => {
  session = fixtures.callers[role]
  vueProps.$localStorage.get.mockImplementation(key => {
    return key === CURRENT_PROJECT && selectedProjectId ? { id: selectedProjectId } : null
  })
  const response = new Promise((resolve, reject) => {
    interceptor = axios.interceptors.response.use(data => {
      resolve(data)
      return data
    }, error => {
      reject(error)
      return Promise.reject(error)
    })
  })
  wrapper = common.createFactory(VolumesTab, {
    router,
    i18n,
    props: { resource: fixture.resource }
  })
  await response
  await flushPromises()

  const ids = (wrapper.vm.volumes || []).map(volume => volume.id).sort()
  console.log(`${name} as ${role}: sent projectid=${sent.projectid} -> rendered ${ids.length} volumes; expected ${fixture.volumeIds.length}`)
  expect(ids).toEqual([...fixture.volumeIds].sort())
  expect(wrapper.findAll('tbody tr[data-row-key]').length).toBe(fixture.volumeIds.length)
}, 60000)

How did you try to break this feature and the system with this change?

I opened the tab as a root admin and as a domain admin, for an instance owned by another user, for a project the domain admin belongs to, for a project they do not belong to, and for the secondary storage system VM, with and without another project selected in the project view.

[boring-cyborg]

Congratulations on your first Pull Request and welcome to the Apache CloudStack community! If you have any issues or are unsure about any anything please check our Contribution Guide (https://github.com/apache/cloudstack/blob/main/CONTRIBUTING.md)
Here are some useful points:

• In case of a new feature add useful documentation (raise doc PR at https://github.com/apache/cloudstack-documentation)
• Be patient and persistent. It might take some time to get a review or get the final approval from the committers.
• Pay attention to the quality of your code, ensure tests are passing and your PR doesn't have conflicts.
• Please follow ASF Code of Conduct for all communication including (but not limited to) comments on Pull Requests, Issues, Mailing list and Slack.
• Be sure to read the CloudStack Coding Conventions.
  Apache CloudStack is a community-driven project and together we are making it better 🚀.
  In case of doubts contact the developers at:
  Mailing List: dev@cloudstack.apache.org (https://cloudstack.apache.org/mailing-lists.html)
  Slack: https://apachecloudstack.slack.com/

[RosiKyu]

@1fanwang could you please request a review for this PR so we can potentially review,test and get this into 24.0?

[1fanwang]

@1fanwang could you please request a review for this PR so we can potentially review,test and get this into 24.0?

@RosiKyu I tried, but GitHub only lets committers request reviewers on apache/cloudstack, so the request fails from my account. Could you add yourself or whoever owns the UI? The PR is ready, and the test and its HTTP fixture are in the diff.

[Damans227]

can we send the vm's own projectid only when it has one? with -1 a domain admin looking at another user's vm in their domain gets an empty tab

[Damans227]

that covers it, the tab now sends the vm's own project in 7e0ef8e