add procedures procedure by DaanHoogland · Pull Request #9385 · apache/cloudstack

DaanHoogland · 2024-07-12T13:33:41Z

Description

This PR...

Types of changes

Breaking change (fix or feature that would cause existing functionality to change)
New feature (non-breaking change which adds functionality)
Bug fix (non-breaking change which fixes an issue)
Enhancement (improves an existing feature and functionality)
Cleanup (Code refactoring and cleanup, that may add test cases)
build/CI
test (unit or integration test code)

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

Major
Minor

Bug Severity

Screenshots (if appropriate):

How Has This Been Tested?

How did you try to break this feature and the system with this change?

codecov · 2024-07-12T13:43:30Z

Codecov Report

Attention: Patch coverage is 1.23457% with 80 lines in your changes missing coverage. Please review.

Project coverage is 15.07%. Comparing base (54c8b71) to head (2925511).

Files	Patch %	Lines
...java/com/cloud/upgrade/DatabaseUpgradeChecker.java	1.23%	80 Missing ⚠️

Additional details and impacted files

@@             Coverage Diff              @@
##               4.19    #9385      +/-   ##
============================================
- Coverage     15.07%   15.07%   -0.01%     
  Complexity    11169    11169              
============================================
  Files          5405     5405              
  Lines        472677   472708      +31     
  Branches      60049    61718    +1669     
============================================
+ Hits          71269    71271       +2     
- Misses       393479   393509      +30     
+ Partials       7929     7928       -1

Flag	Coverage Δ
uitests	`4.27% <ø> (ø)`
unittests	`15.79% <1.23%> (-0.01%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

blueorangutan · 2024-07-12T15:16:03Z

@DaanHoogland a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

GutoVeronezi · 2024-07-12T15:59:39Z

Nice work @DaanHoogland!

~~I believe we could do the same for the cloud_usage stored procedures.~~

Should we separate the cloud_usage in different files?

blueorangutan · 2024-07-12T16:20:49Z

Packaging result [SF]: ✔️ el7 ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 10344

DaanHoogland · 2024-07-13T19:34:25Z

Nice work @DaanHoogland!

~~I believe we could do the same for the cloud_usage stored procedures.~~

Should we separate the cloud_usage in different files?

I thought of that but wanted to check versions against each other. I will make that change.

A change I won't make but what we need to do in another PR is cleaning the views from the schema update files. I encoutered a couple.

DaanHoogland · 2024-07-13T19:35:01Z

@blueorangutan test

blueorangutan · 2024-07-13T19:36:03Z

@DaanHoogland a [SL] Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

blueorangutan · 2024-07-14T08:01:58Z

[SF] Trillian test result (tid-10821)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 43258 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr9385-t10821-kvm-centos7.zip
Smoke tests completed. 131 look OK, 1 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test	Result	Time (s)	Test File
test_01_condensed_drs_algorithm	`Failure`	173.39	test_cluster_drs.py

DaanHoogland · 2024-07-14T13:59:46Z

@blueorangutan test matrix

blueorangutan · 2024-07-14T14:02:03Z

@DaanHoogland a [SL] Trillian-Jenkins matrix job (centos7 mgmt + xenserver71, rocky8 mgmt + vmware67u3, centos7 mgmt + kvmcentos7) has been kicked to run smoke tests

blueorangutan · 2024-07-15T03:00:12Z

[SF] Trillian test result (tid-10828)
Environment: xenserver-71 (x2), Advanced Networking with Mgmt server 7
Total time taken: 45251 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr9385-t10828-xenserver-71.zip
Smoke tests completed. 132 look OK, 0 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test	Result	Time (s)	Test File

DaanHoogland · 2024-07-15T08:38:54Z

@blueorangutan package

blueorangutan · 2024-07-15T08:40:03Z

@DaanHoogland a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

blueorangutan · 2024-07-15T09:47:07Z

Packaging result [SF]: ✔️ el7 ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 10364

DaanHoogland · 2024-07-15T10:05:37Z

@blueorangutan test matrix

blueorangutan · 2024-07-15T10:06:04Z

@DaanHoogland a [SL] Trillian-Jenkins matrix job (centos7 mgmt + xenserver71, rocky8 mgmt + vmware67u3, centos7 mgmt + kvmcentos7) has been kicked to run smoke tests

blueorangutan · 2024-07-15T10:25:38Z

[SF] Trillian Build Failed (tid-10849)

GutoVeronezi · 2024-07-15T12:33:52Z

Nice work @DaanHoogland!
~~I believe we could do the same for the cloud_usage stored procedures.~~
Should we separate the cloud_usage in different files?

I thought of that but wanted to check versions against each other. I will make that change.

@DaanHoogland
I was thinking a little bit more about this, and, except for ADD_GUEST_OS_AND_HYPERVISOR_MAPPING, the other procedures are valid for both cloud and cloud_usage. Perhaps we could execute the script once for each schema instead of duplicating the code. What do you think?

DaanHoogland · 2024-07-15T12:38:35Z

Nice work @DaanHoogland!
~~I believe we could do the same for the cloud_usage stored procedures.~~
Should we separate the cloud_usage in different files?

I thought of that but wanted to check versions against each other. I will make that change.

@DaanHoogland I was thinking a little bit more about this, and, except for ADD_GUEST_OS_AND_HYPERVISOR_MAPPING, the other procedures are valid for both cloud and cloud_usage. Perhaps we could execute the script once for each schema instead of duplicating the code. What do you think?

that sounds good, but so far creating envs with this code doesn't work. I am not sure why yet, but giving that priority.

(will keep it in mind though)

DaanHoogland · 2024-07-17T09:04:23Z

@blueorangutan package

blueorangutan · 2024-07-17T09:06:04Z

@DaanHoogland a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

blueorangutan · 2024-07-17T10:10:55Z

Packaging result [SF]: ✔️ el7 ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 10386

blueorangutan · 2024-07-17T11:55:41Z

[SF] Trillian Build Failed (tid-10900)

blueorangutan · 2024-07-18T09:34:38Z

[SF] Trillian Build Failed (tid-10904)

blueorangutan · 2024-07-18T13:31:10Z

[SF] Trillian Build Failed (tid-10908)

blueorangutan · 2024-07-19T07:48:12Z

[LL] Trillian Build Failed (tid-6958)

blueorangutan · 2024-07-19T09:37:20Z

[SF] Trillian Build Failed (tid-10918)

blueorangutan · 2024-07-19T11:13:27Z

[SF] Trillian Build Failed (tid-10921)

blueorangutan · 2024-07-19T14:37:32Z

Packaging result [SF]: ✔️ el7 ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 10415

blueorangutan · 2024-07-20T07:07:37Z

[SF] Trillian test result (tid-10927)
Environment: kvm-alma8 (x2), Advanced Networking with Mgmt server a8
Total time taken: 52176 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr9385-t10927-kvm-alma8.zip
Smoke tests completed. 131 look OK, 1 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test	Result	Time (s)	Test File
test_05_rvpc_multi_tiers	`Failure`	527.03	test_vpc_redundant.py
test_05_rvpc_multi_tiers	`Error`	527.05	test_vpc_redundant.py

DaanHoogland · 2024-07-22T08:31:20Z

@GutoVeronezi i got this working and did some investigating on iterating over databases but all solutions so far seem overcomplicated. Let's revisist later, ok?

blueorangutan · 2024-07-22T23:54:12Z

[SF] Trillian test result (tid-10936)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 43631 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr9385-t10936-kvm-centos7.zip
Smoke tests completed. 132 look OK, 0 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test	Result	Time (s)	Test File

weizhouapache

code lgtm

just a tip for those who want to create the procedures manually
the SQLs do not work in mysql CLI as the default delimiter is ";", need to change to other value like "||"

DROP PROCEDURE IF EXISTS ...

DELIMITER ||

CREATE PROCEDURE ...
||

DELIMITER ;

vladimirpetrov

LGTM based on manual testing, I did some upgrade and smoke testing and found no issues.

DaanHoogland force-pushed the procedures branch from 541e76c to 3167dc4 Compare July 12, 2024 15:15

DaanHoogland force-pushed the procedures branch from 90eed12 to 788b5cd Compare July 15, 2024 08:38

DaanHoogland force-pushed the procedures branch from 788b5cd to 257a26e Compare July 15, 2024 11:14

apache deleted a comment from blueorangutan Jul 16, 2024

DaanHoogland force-pushed the procedures branch 2 times, most recently from c20be12 to bd3512f Compare July 19, 2024 13:27

DaanHoogland marked this pull request as ready for review July 22, 2024 07:47

add procedures procedure

2925511

DaanHoogland force-pushed the procedures branch from bd3512f to 2925511 Compare July 22, 2024 07:48

weizhouapache mentioned this pull request Jul 22, 2024

Fix 4.17.2 upgrade schema idempotent add column order #9427

Closed

14 tasks

DaanHoogland requested review from GutoVeronezi, nvazquez, vladimirpetrov and weizhouapache July 23, 2024 09:40

weizhouapache approved these changes Jul 23, 2024

View reviewed changes

yadvr approved these changes Jul 29, 2024

View reviewed changes

vladimirpetrov approved these changes Aug 7, 2024

View reviewed changes

DaanHoogland merged commit 6e6a276 into apache:4.19 Aug 12, 2024

DaanHoogland deleted the procedures branch August 12, 2024 12:47

dhslove pushed a commit to ablecloud-team/ablestack-cloud that referenced this pull request Aug 22, 2024

add procedures procedure (apache#9385)

d252607

DaanHoogland added this to the 4.19.2.0 milestone Sep 6, 2024

Conversation

DaanHoogland commented Jul 12, 2024

Description

Types of changes

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

Bug Severity

Screenshots (if appropriate):

How Has This Been Tested?

How did you try to break this feature and the system with this change?

Uh oh!

codecov bot commented Jul 12, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

blueorangutan commented Jul 12, 2024

Uh oh!

GutoVeronezi commented Jul 12, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

blueorangutan commented Jul 12, 2024

Uh oh!

DaanHoogland commented Jul 13, 2024

Uh oh!

DaanHoogland commented Jul 13, 2024

Uh oh!

blueorangutan commented Jul 13, 2024

Uh oh!

blueorangutan commented Jul 14, 2024

Uh oh!

DaanHoogland commented Jul 14, 2024

Uh oh!

blueorangutan commented Jul 14, 2024

Uh oh!

blueorangutan commented Jul 15, 2024

Uh oh!

DaanHoogland commented Jul 15, 2024

Uh oh!

blueorangutan commented Jul 15, 2024

Uh oh!

blueorangutan commented Jul 15, 2024

Uh oh!

DaanHoogland commented Jul 15, 2024

Uh oh!

blueorangutan commented Jul 15, 2024

Uh oh!

blueorangutan commented Jul 15, 2024

Uh oh!

GutoVeronezi commented Jul 15, 2024

Uh oh!

DaanHoogland commented Jul 15, 2024

Uh oh!

DaanHoogland commented Jul 17, 2024

Uh oh!

blueorangutan commented Jul 17, 2024

Uh oh!

blueorangutan commented Jul 17, 2024

Uh oh!

blueorangutan commented Jul 17, 2024

Uh oh!

blueorangutan commented Jul 18, 2024

Uh oh!

blueorangutan commented Jul 18, 2024

Uh oh!

blueorangutan commented Jul 19, 2024

Uh oh!

blueorangutan commented Jul 19, 2024

Uh oh!

blueorangutan commented Jul 19, 2024

Uh oh!

blueorangutan commented Jul 19, 2024

Uh oh!

blueorangutan commented Jul 20, 2024

Uh oh!

DaanHoogland commented Jul 22, 2024

Uh oh!

blueorangutan commented Jul 22, 2024

Uh oh!

weizhouapache left a comment

Choose a reason for hiding this comment

codecov bot commented Jul 12, 2024 •

edited

Loading

GutoVeronezi commented Jul 12, 2024 •

edited

Loading