Currently there is fake rpc call "pod_health_probe_method_ignore_errors"
that is passed to the service, just to find out if it is responding. Because
such method does not exist, it is needed to catch and handle the exception
that is inevitably thrown by the service.
While this is technically working correctly, the exceptions pollute the
log files and make it harder for user to see possible real errors.
This is how the error looks like:
ERROR oslo_messaging.rpc.server [-] Exception during message handling: oslo_messaging.rpc.dispatcher.UnsupportedVersion: Endpoint does not support RPC version 1.0. Attempted method: pod_health_probe_method_ignore_errors
ERROR oslo_messaging.rpc.server Traceback (most recent call last):
ERROR oslo_messaging.rpc.server File "/var/lib/openstack/lib/python3.6/site-packages/oslo_messaging/rpc/server.py", line 163, in _process_incoming
ERROR oslo_messaging.rpc.server res = self.dispatcher.dispatch(message)
ERROR oslo_messaging.rpc.server File "/var/lib/openstack/lib/python3.6/site-packages/oslo_messaging/rpc/dispatcher.py", line 276, in dispatch
ERROR oslo_messaging.rpc.server raise UnsupportedVersion(version, method=method)
ERROR oslo_messaging.rpc.server oslo_messaging.rpc.dispatcher.UnsupportedVersion: Endpoint does not support RPC version 1.0. Attempted method: pod_health_probe_method_ignore_errors
This situation is new since https://review.openstack.org/#/c/639711/
which (correctly) increased the default level of logging. Before 639711
error messages from oslo (both real and ones that could be ignored) were not
present in nova logs at all.
Fortunatelly, nova's BaseAPI class provides 'ping' method that is can
be used for this basic purpose by all nova components.
Change-Id: I0062e74bed399206becb8d9e00f9ec805da864a3
This PS adds emptydirs backing the /tmp directory in pods, which
is required in most cases for full operation when using a read only
filesystem backing the container.
Additionally some yaml indent issues are resolved.
Change-Id: I9df8f70e913b911ff755600fa2f669d9c5dcb928
Signed-off-by: Pete Birley <pete@port.direct>
When novnc pod is re-run because of host reboot and so on,
novnc pod has existing volume /tmp/usr/share, which has 0444 permissions.
So init container occurs an error while it tries to copy asset files.
cp: cannot create regular file '/tmp/usr/share/novnc/index.html': Permission denied
With -f option, the init container can copy without errors.
Change-Id: I56d928b7f4a30a6be29b47560357a3b4f5eec764
Signed-off-by: hagun.kim <hagun.kim@samsung.com>
With this patch we allow for a more easy way of overriding some of
the values that may be used in other distros while maintainting the
default values if those values are not overriden
The following values are introduced to be overriden:
conf:
security:
software:
apache2:
conf_dir:
site_dir:
mods_dir
binary:
extra_flags:
a2enmod:
a2dismod:
On which:
* conf_dir: directory where to drop the config files for apache vhosts
* site_dir: directory where to drop the enabled virtualhosts
* mods_dir: directory where to drop any mod configuration
* binary: the binary to use for launching apache
* extra_flags: any flags that will be passed to the apache binary call
* a2enmod: mods to enable
* a2dismod: mods to disable
* security: security configuration for apache
Notice that if there is no overrides given, it should not affect anything
and the templates will not be changed as the default values are set
to what they used to be
Change-Id: I4fcfde78c5c8fa65956aeae55108ffa1f10e6972
This change adds the keystonemiddleware audit paste filter[0]
and enables it for the nova-api services.
This provides the ability to audit API requests for nova.
[0] https://docs.openstack.org/keystonemiddleware/latest/audit.html
Change-Id: Ic6df044d83f4dee581c9cc0405f61d926e45bcab
Using {{- if for the volume mounts caused them to be added inline with
the previous line.
Removing the - from the if expression makes them be properly aligned on
the next line
Change-Id: Ia5e28366fb1f2ae7420b7f5217c10cbb94bc48ab
Only start the sshd container of nova-compute pod if the capability is
enabled. Defaults to off to allow cases where nova docker image does
not have ssh packages to run cleanly.
Story: 2003463
Task: 30441
Change-Id: I3acf5b654ecda23a93f6c28e865e1bbee14370aa
Signed-off-by: Gerry Kopec <Gerry.Kopec@windriver.com>
- Fix .ssh/config file mapping
- Move private key from nova-compute-ssh container to nova-compute
container.
- Map private and public keys to configmap-ssh which will default to
the appropriate file permissions.
- Add additional config to /etc/ssh/sshd_config to allow passwordless
root logins over appropriate subnet passed in from overrides.
- Remove chmods from sshd bash script as they are failing.
Depends on helm-toolkit supporting multiple containers per daemonset
pod.
Story: 2003463
Task: 24723
Change-Id: Idd2e802c293f1e14991ee787ade9a4936fb373ff
Signed-off-by: Gerry Kopec <Gerry.Kopec@windriver.com>
On some services it looks like the parent pid does not connect to
rabbitmq and its the children the ones that do instead, for example
in nova-scheduler from rocky version onwards.
The current health check only checks for the main parent pid to see
if it has an active connection to the rabbitmq port.
This patch adds a flag to allow the health probe to check all processes
for the mysql/rabbit connection instead of skipping any children process.
It also enables it by default for nova-scheduler as it wont affect older versions
than only run 1 process, but will work on later versions where
the main process forks.
Change-Id: I9677fd2aff11b563ab18059927ca12d5ace107ce
Under python3 an Exception no longer has the message attribute,
instead you can just str the exception to get the error message
Change-Id: Ibf88ae6b73f3bafcc2b99bb01e31bf8c25021e47
Health probe for Nova pods is used for both liveness
and readiness probe.
nova-compute, nova-conductor, nova-consoleauth and nova-scheduler:
Check if the rpc socket status on the nova pods to rabbitmq and
database are in established state.
sends an RPC call with a non-existence method to component's queue.
Probe is success if agent returns with NoSuchMethod error.
If agent is not reachable or fails to respond in time,
returns failure to probe.
novnc/spice proxy: uses Kubernetes tcp probe on corresponding ports
they expose.
Added code to catch nova config file not present exception.
Change-Id: Ib8e4b93486588320fd2d562c3bc90b65844e52e5
This patch set adds "startingDeadlineSeconds" field to cronJobs.
When the field is not set, the controller counts how many missed
jobs occured from the last scheduled time till now. And if it happends
more than 100 time the job will not be scheduled. To avoid this
the "startingDeadlineSeconds" field should be set to sufficient period
of time. In this case the controller counts how many missed jobs occured
during this period of time. The value of the field should be less than
time (in seconds) needed for running >100 jobs (according to schedule).
Change-Id: I3bf7c7077b55ca5a3421052bd0b59b70c9bbcf24
This adds the release-uuid annotation to the pod spec for all
replication controller templates in the openstack-helm charts
Change-Id: I0159f2741c27277fd173208e7169ff657bb33e57
This patch set host_interface for update host_ip information in compute
node.
Currently helm chart defines the value of my_ip set "0.0.0.0",
therefore host_ip of compute node is null.
$ nova hypervisor-show {uuid}
+---------------------------+------------------------------------------+
| Property | Value |
+---------------------------+------------------------------------------+
| cpu_info_arch | x86_64 |
.
.
| host_ip | None |
Through this patch, OpenStack can provide appropriate values for
the required field.
Change-Id: I05f929cb2c777582c177e8c7a64b9fd431d554ec
This patchset enables and moves the securityContext: runAsUser to the pod
level, and uses a non-root user (UID != 0) wherever applicable.
Depends-On: I95264c933b51e2a8e38f63faa1e239bb3c1ebfda
Change-Id: I81f6e11fe31ab7333a3805399b2e5326ec1e06a7
Signed-off-by: Tin Lam <tin@irrational.io>
This PS updates the mount options for the nova-compute pod to mount
cgroups as read only within the pod.
Change-Id: I82e958c2865029cd4a093f62614a1e878075098a
Signed-off-by: Pete Birley <pete@port.direct>
The update makes sure the Openstack service's cephx
user capabilities match best practices in terms of
security permissions after a site or software update.
Change-Id: I70e7f620accb186da2013ba95472777c25739cc1
This patch set updates the gate to by default uses network policy
for all components and enforces them in Openstack-helm.
Change-Id: I70c90b5808075797f02670f21481a4f968205325
Depends-On: I78e87ef3276e948ae4dd2eb462b4b8012251c8c8
Co-Authored-By: Mike Pham <tp6510@att.com>
Signed-off-by: Tin Lam <tin@irrational.io>
If conf.nova.ironic.memcache_secret_key is not explicitly set, derive
it from endpoints.oslo_cache.auth.memcache_secret_key or use a random
value.
This means when installing charts where we explicitly set
endpoints.oslo_cache.auth.memcache_secret_key upgrades without changes
won't churn.
Change-Id: I686297e25627d88ff9bd32df0a3f7ee8afc11f58
While implementing network policies, we noticed a handful of pods created
as part of a CronJobs are missing labels causing them to be unable to
targed by the policy. This patch set adds in the missing labels found
in that effort.
Change-Id: I1ca3cfd68ff20dc39a1e952414f3dddd3fc8d3b4
This PS adds the ability to attach a release uuid to pods and rc
objects as desired. A follow up ps will add the ability to add arbitary
annotations to the same objects.
Depends-On: Iceedba457a03387f6fc44eb763a00fd57f9d84a5
Change-Id: I324680f10263c1aefca2be9056e70d0ff22fcaf0
Signed-off-by: Pete Birley <pete@port.direct>
This is make ceph configmap and admin keyring secret names using
in storage init scripts to be read from chart values as we may
have two ceph clusters gets activated in one namespace and
each ceph clsuter will have its own configmap and admin secret names.
Change-Id: I84d94f3ac21e602c50619e456ff327ae1da53622
This PS moves openstack components in OSH to use secrets to store
potentially sensitive config information.
Depends-On: https://review.openstack.org/#/c/593732
Change-Id: I9bab586c03597effea0e48a58c69efff3f980a92
Signed-off-by: Pete Birley <pete@port.direct>
This PS moves nova inline with other charts, and drives all config
directly from the charts values.yaml.
Change-Id: Ia3da97cd32e70e3a5ffe0ed3f3cacfbadfc8cfd6
Signed-off-by: Pete Birley <pete@port.direct>
Some environment which is enabled zeroconf network, it returns
multilines. This PS make to get the one ip address correctly.
Change-Id: I577f02908b76b280d8fa87acec25d96c3f556e47
This PS updates the nova rootwrap config to be values.yaml driven,
inline with neutron and cinder.
Change-Id: I3df7e7de583a75234a6bc4a71a32bd0a8d369332
Signed-off-by: Pete Birley <pete@port.direct>
This PS brings novas sudoers file config to be in line with neutron
and other osh charts.
Change-Id: I609455b1a58e576644b8a6cfb8b273d5aeb94c58
Signed-off-by: Pete Birley <pete@port.direct>
