feature: Add Public API Server Address Support#986
Open
rossigee wants to merge 26 commits into
Open
Conversation
✅ Deploy Preview for kamaji-documentation canceled.
|
prometherion
requested changes
Oct 10, 2025
prometherion
requested changes
Oct 18, 2025
prometherion
left a comment
prometherion
left a comment
Member
There was a problem hiding this comment.
I'm a bit lost here: it seems the proposed public API Server address is not pushed into the kubeadm functions, such as:
kamaji/internal/resources/kubeadm_config.go
Line 106 in 99fb71e
and the following one for kubeconfig
kamaji/internal/resources/kubeconfig.go
Line 234 in 99fb71e
Comment on lines
+20
to
+26
| // Fall back to the assigned control plane address, but use configured port | ||
| assignedAddress, _, err := in.AssignedControlPlaneAddress() | ||
| if err != nil { | ||
| return "", 0, err | ||
| } | ||
|
|
||
| return assignedAddress, port, nil |
Member
There was a problem hiding this comment.
Suggested change
| // Fall back to the assigned control plane address, but use configured port | |
| assignedAddress, _, err := in.AssignedControlPlaneAddress() | |
| if err != nil { | |
| return "", 0, err | |
| } | |
| return assignedAddress, port, nil | |
| // Fall back to the assigned control plane address | |
| return in.AssignedControlPlaneAddress() |
Unless I'm missing something, there's no need to replicate the same behavior of AssignedControlPlaneAddress.
Comment on lines
+223
to
+231
| // Post-process the kubeconfig to use the public API server address for | ||
| // controller manager and scheduler components instead of localhost/IP address | ||
| if strings.Contains(r.KubeConfigFileName, "controller-manager") || strings.Contains(r.KubeConfigFileName, "scheduler") { | ||
| kubeconfig, kcErr = r.replaceServerURLWithPublicAddress(kubeconfig, tenantControlPlane) | ||
| if kcErr != nil { | ||
| logger.Error(kcErr, "cannot replace server URL in kubeconfig") | ||
| return kcErr | ||
| } | ||
| } |
Member
There was a problem hiding this comment.
Unless I'm missing something, this is not required: Kamaji leverages the concept of using loopback to communicate with the API Server; otherwise, we would end up with extra network hops with potential egress traffic.
Comment on lines
+272
to
+280
| func (r *KubeconfigResource) usePublicAPIServerAddress(config *kubeadm.Configuration, tenantControlPlane *kamajiv1alpha1.TenantControlPlane) error { | ||
| // Use the public API server endpoint configured in the TenantControlPlane | ||
| // instead of localhost for controller manager and scheduler kubeconfigs. | ||
| // This ensures that internal control plane components connect using the proper hostname | ||
| // which matches the certificate SANs, rather than failing with x509 certificate errors. | ||
|
|
||
| // Note: We don't modify the kubeadm configuration here because LocalAPIEndpoint.AdvertiseAddress | ||
| // must be an IP address according to kubeadm validation. Instead, we'll post-process | ||
| // the generated kubeconfig to replace the server URL with the hostname. |
Member
There was a problem hiding this comment.
Same discussion as we said before: Scheduler and Controller Manager must rely on the loopback address.
Member
|
@rossigee I'm sorry the rebase will be painful, let me know if I can take care of that! |
rossigee
force-pushed
the
feature/public-address
branch
from
2bea3f5 to
eaffca1
Compare
✅ Deploy Preview for kamaji-documentation canceled.
|
rossigee
force-pushed
the
feature/public-address
branch
from
922610a to
566d3c9
Compare
Add PublicAPIServerAddress field to ServiceSpec for specifying custom API server hostnames. Implement PublicControlPlaneAddress method to return public address with fallback to assigned address. Integrate public address usage in kubeconfig generation for controller-manager and scheduler. Add comprehensive tests for the functionality.
Add PublicAPIServerAddress field to TenantControlPlane ServiceSpec to allow specifying a custom hostname for the API server. This enables using DNS names that match certificate SANs instead of LoadBalancer IPs. Key changes: - Add PublicAPIServerAddress field to ServiceSpec - Implement PublicControlPlaneAddress method with fallback logic - Update CRDs and API documentation - Remove kubeconfig post-processing for internal components (scheduler/controller-manager) - Use controller-gen v0.16.1 as per project standards The public address is used for external access (cluster-info ConfigMaps and user kubeconfigs), while internal components continue to use loopback addresses.
…nfigs - Add PublicControlPlaneAddress method to prioritize public address over assigned address - Update kubeadm config generation to use public address for ControlPlaneEndpoint - Automatically include public address in certificate SANs if not present - Enhance e2e tests to validate kubeconfig server URLs use public address - Fix lint issues and format code This enables DNS-based cluster-info and kubeconfigs when PublicAPIServerAddress is specified.
…r-manager and scheduler kubeconfigs - Controller-manager and scheduler now use cluster-local svc addresses in kubeconfigs - Public address is still used for admin kubeconfigs and cluster-info - Updated e2e tests to validate local endpoints for in-cluster components
- Update DeclaredControlPlaneAddress to return clusterIP when available for kubeadm compatibility - Add nil check in worker kubeadm join test to prevent panic - Fix import ordering in public address test
Add nil check for stdout before io.ReadAll to prevent panic.
Check for nil data and config fields before accessing them.
Check for AlreadyExists error when creating GatewayClass to allow reruns.
Add nil check for stdout before io.ReadAll.
Check if cluster exists in config before accessing it.
This avoids panics and provides clear failure messages.
Use context.Cluster to get the cluster name from kubeconfig.
Update customizeConfig to use full DNS with cluster.local, and update test expectations.
Update customizeConfig and test expectations to match original behavior.
Update expectations to match the public address when set.
…ager and scheduler kubeconfigs Only override to internal DNS if PublicAPIServerAddress is not set.
Kubeconfig cluster name is always 'kubernetes' in the Clusters map.
The cluster key is context.Cluster, not hardcoded 'kubernetes'.
The kubeconfig cluster name is always 'kubernetes'.
The kubeconfig cluster name is set to the TCP name.
The exec may return err even when command succeeds, check exitCode for proper failure detection.
The kubeconfig cluster name is always 'kubernetes'.
This should work regardless of the cluster name.
Return PublicAPIServerAddress if set, for proper endpoint configuration.
- Always use cluster-local Service addresses for internal components - Avoid unnecessary network hops through external LoadBalancer - Update tests to verify local addresses instead of public address
rossigee
force-pushed
the
feature/public-address
branch
from
d3e5503 to
44e305e
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Add Public API Server Address Support
Overview
This PR introduces support for specifying a custom public API server address for TenantControlPlane instances. This allows using DNS hostnames in cluster-info ConfigMaps and kubeconfigs instead of LoadBalancer IPs, enabling better certificate SAN matching and avoiding x509 certificate errors.
Key Changes
API Changes
PublicAPIServerAddressfield toServiceSpecin the TenantControlPlane APIImplementation
PublicControlPlaneAddress()method toTenantControlPlanestructcmp.Orfor default port (6443)Tests
PublicControlPlaneAddress()methodDocumentation
Usage Example
Benefits
Breaking Changes
None. The field is optional and defaults to existing behavior.
Related Issues