From 3.2.2 to 3.3.0
Summary
Optimization Server 3.3.0 introduces break changes in:
the Worker Java library
It also deprecates application properties used in the workers (both Java and Python)
You are concerned by this guide if :
you develop your application, client of Optimization Server. → Read this section.
you develop your own Java worker. → Read this section.
you develop your own Python worker. → Read this section).
you deploy the worker (cplex or wod) Helm charts. → Read this section
you deploy your own worker Helm chart. → Read this section
Development of applications
Library | Backward compatibility | Deprecations |
---|---|---|
Master API client | Break changes: adjust code | |
Worker (Java) | Break changes: adjust code | Deprecations |
Worker (Python) | Backward compatible | Deprecations |
Deployment of Optimization Server
Chart | Backward compatibility | Deprecations |
---|---|---|
dbos-volume | Backward compatible | |
dbos-secrets | Backward compatible | |
dbos-infra | Backward compatible | |
dbos | Backward compatible | |
cplex | Backward compatible | Deprecations |
wod | Backward compatible | Deprecations |
Master API client
For the synchronous client, please read this section
For the asynchronous clients, read:
The synchronous API client
The former Java clients:
jersey
rest template
feign
have been merged into one single optimserver-api-client
Java library.
Optimization Server 3.3.0 also provides a Spring Boot starter module if you want to integrate the library in a Spring Boot application.
Read this section for mode details about the new Java library.
Read this one for mode details about the Spring integration.
Even though most of the code from the former clients is still functional, some of it must be adjusted.
The import statements
The sub-packages jersey2
, feign
and resttemplate
have disappeared.
3.2.2:
import com.decisionbrain.optimserver.client.java.jersey2.ApiClient;
import com.decisionbrain.optimserver.client.java.feign.ApiClient;
import com.decisionbrain.optimserver.client.java.resttemplate.ApiClient;
import com.decisionbrain.optimserver.client.java.jersey2.model.JobStatusEvent;
import com.decisionbrain.optimserver.client.java.feign.model.JobStatusEvent;
import com.decisionbrain.optimserver.client.java.resttemplate.model.JobStatusEvent;
3.3.0:
import com.decisionbrain.optimserver.client.java.ApiClient;
import com.decisionbrain.optimserver.master.model.JobStatusEvent;
The 'File' type
The APIs that used File
as the return type now return InputStream
3.2.2:
BucketApi api;
String bucketId = "id";
...
File bucketFile = api.getBucketContent(bucketId);
try(FileInputStream fileInputStream = new FileInputStream(bucketFile)){
byte[] bucketData = fileInputStream.readAllBytes();
}
3.3.0:
BucketApi api;
String bucketId = "id";
....
try(InputStream bucketStream = api.getBucketContent(bucketId)){
byte[] bucketData = bucketStream.readAllBytes();
}
The Spring AMQP Asynchronous API client
The Asynchronous API has been fully rewritten in Optimization Server 3.3.0.
3.2.2:
implementation "com.decisionbrain:optimserver-client-amqp-spring:3.2.2"
@SpringBootApplication
@EnableOptimServerAmqpClient
public class ApiAmqpClientApplication {
public static void main(String[] args) {
SpringApplication.run(ApiAmqpClientApplication.class, args);
}
}
import com.decisionbrain.optimserver.client.amqp.service.JobsCompletionWaitingService;
import com.decisionbrain.optimserver.client.amqp.service.JobEventsStandaloneSubscriptionService;
import com.decisionbrain.optimserver.client.java.jersey2.api.JobExecutionApi;
@Service
public class MyService
{
private final JobsCompletionWaitingService jobsCompletionWaitingService;
private final JobEventsStandaloneSubscriptionService jobEventsListenerRegisteringService;
private final JobExecutionApi jobExecutionClient;
public MyService(JobsCompletionWaitingService jobsCompletionWaitingService,
JobEventsStandaloneSubscriptionService jobEventsListenerRegisteringService,
JobExecutionApi jobExecutionClient) {
this.jobsCompletionWaitingService = jobsCompletionWaitingService;
this.jobEventsListenerRegisteringService = jobEventsListenerRegisteringService;
this.jobExecutionClient = jobExecutionClient;
}
public void listenToOptimServerJobSolution() {
String jobId = "created job Id";
long timeout = 5000L;
jobsCompletionWaitingService.waitForJobCompletion(jobId, timeout * 2, () -> {
return jobExecutionClient.startAsyncJobExecution(jobId, timeout) != null;
}).ifPresent((SolutionDTO solution) -> {
LOGGER.info("Solution retrieved before timeout");
});
}
void listenToOptimServerJobEvents() {
String jobId = "created job Id";
jobEventsListenerRegisteringService.registerJobEventsListener(jobId, new JobEventListener() {
@Override
public void onStatus(JobStatusDTO jobStatus) {
LOGGER.info(String.format("[STATUS]: %s", jobStatus.getStatus().name()));
}
});
}
}
3.3.0:
implementation "com.decisionbrain:spring-boot-starter-optimserver-amqp-client:3.3.0"
@SpringBootApplication
@EnableOptimServerClient
public class ApiAmqpClientApplication {
public static void main(String[] args) {
SpringApplication.run(ApiAmqpClientApplication.class, args);
}
}
import com.decisionbrain.optimserver.client.java.async.api.JobExecutionAsyncApi;
import com.decisionbrain.optimserver.client.java.async.api.JobEventSource;
import com.decisionbrain.optimserver.client.java.api.JobExecutionApi;
import java.time.Duration;
@Service
public class MyService {
private final JobExecutionAsyncApi jobExecutionAsyncApi;
private final JobExecutionApi jobExecutionClient;
public MyService(JobExecutionAsyncApi jobExecutionAsyncApi, JobExecutionApi jobExecutionClient) {
this.jobExecutionAsyncApi = jobExecutionAsyncApi;
this.jobExecutionClient = jobExecutionClient;
}
public void listenToOptimServerJobSolution() {
String jobId = "created job Id";
long timeout = 5000L;
jobExecutionAsyncApi.getJobSolution(jobId, Duration.ofMillis(timeout * 2))
.whenComplete((jobSolution, throwable) -> {
if (jobSolution != null) {
LOGGER.info("Solution retrieved before timeout");
}
});
jobExecutionApi.startAsyncJobExecution(jobDefinition.getId(), timeout);
}
void listenToOptimServerJobEvents() {
String jobId = "created job Id";
JobSubscriptionFilter filter = new JobSubscriptionFilter(jobId);
JobEventSource eventSource = jobExecutionAsyncApi.getJobEventSource(filter);
eventSource.statusEvents()
.subscribe(new ConsumerSubscriber<>(
jobStatusEvent -> LOGGER.info(String.format("[STATUS]: %s", jobStatusEvent.getStatus().name()))
));
eventSource.connect();
}
}
The Spring SSE Asynchronous API client
The Asynchronous API has been fully rewritten in Optimization Server 3.3.0.
3.2.2:
implementation "com.decisionbrain:optimserver-client-sse-spring:3.2.2"
application.yaml
optim-server:
sse:
client:
master:
url: https://master-host/
@SpringBootApplication
@Import(SseSpringConfig.class)
public class ApiSSEClientApplication {
public static void main(String[] args) {
SpringApplication.run(ApiSSEClientApplication.class, args);
}
}
import com.decisionbrain.optimserver.client.sse.SseListener;
@Service
public class MyService
{
private final SseListener sseListener;
void listenToOptimServerJobStatusEvents() {
String jobId = "created job Id";
sseListener.register(jobId, new JobEventListener() {
@Override
public Consumer<JobStatusEvent> onJobStatusEvent() {
return (jobStatusEvent) -> LOGGER.info(String.format("[STATUS]: %s", jobStatusEvent.getStatus().name()));
}
});
}
}
3.3.0:
implementation "com.decisionbrain:spring-boot-starter-optimserver-sse-client:3.3.0"
application.yaml
optim-server:
url: https://master-host/
@SpringBootApplication
@EnableOptimServerClient
public class ApiSSEClientApplication {
public static void main(String[] args) {
SpringApplication.run(ApiSSEClientApplication.class, args);
}
}
import com.decisionbrain.optimserver.client.java.async.api.JobExecutionAsyncApi;
import com.decisionbrain.optimserver.client.java.async.api.JobEventSource;
@Service
public class MyService
{
private final JobExecutionAsyncApi jobExecutionAsyncApi;
public MyService(JobExecutionAsyncApi jobExecutionAsyncApi) {
this.jobExecutionAsyncApi = jobExecutionAsyncApi;
}
void listenToOptimServerJobStatusEvents() {
String jobId = "created job Id";
JobSubscriptionFilter filter = new JobSubscriptionFilter(jobId);
JobEventSource eventSource = jobExecutionAsyncApi.getJobEventSource(filter);
eventSource.statusEvents()
.subscribe(new ConsumerSubscriber<>(
jobStatusEvent -> LOGGER.info(String.format("[STATUS]: %s", jobStatusEvent.getStatus().name()))
));
eventSource.connect();
}
}
The Jersey SSE Asynchronous API client
The Asynchronous API has been fully rewritten in Optimization Server 3.3.0.
3.2.2:
implementation 'com.decisionbrain:optimserver-client-sse-jersey:3.2.2'
public class SampleJavaSSEClient {
private static SseListener createSseClient() {
// This is the main keycloak configuration to authenticate on your Master API.
final KeycloakClientConfig keycloakClientConfig = KeycloakClientConfig.builder()
.url(KEYCLOAK_URL)
.realm(KEYCLOAK_REALM)
.client(KEYCLOAK_CLIENT)
.user(KEYCLOAK_USER)
.password(KEYCLOAK_PASSWORD)
.build();
final AuthenticationService authenticationService = new KeycloakAuthenticationServiceImpl(keycloakClientConfig);
// Configure the master API url and optionally the SSE reconnect time.
final SseJerseyConfig sseJerseyConfig = SseJerseyConfig.builder()
.apiUrl("http://" + DBOS_MASTER_HOST + ":" + DBOS_MASTER_PORT)
.reconnect(2000)
.build();
// Create the listener service instance
return new JerseySseListenerImpl(authenticationService, sseJerseyConfig);
}
public static void main() {
try {
String jobId = "created job Id";
final SseListener seeClient = createSseClient();
seeClient.register(jobDefinition.getId(), new JobEventListener() {
@Override
public Consumer<JobStatusEvent> onJobStatusEvent() {
return (jobStatusEvent) -> LOGGER.info(String.format("[STATUS]: %s", jobStatusEvent.getStatus().name()));
}
@Override
public Consumer<Throwable> onJobStatusEventError() {
return (throwable) -> LOGGER.error("Exception while listening to the job status", throwable);
}
});
jobExecutionClient.startAsyncJobExecution(jobId, null);
LOGGER.info("Waiting for a solution...");
} catch (ApiException e) {
logApiException(e);
} catch (Exception e) {
LOGGER.error("An error occurred: ", e);
}
}
}
3.3.0
implemmentation 'com.decisionbrain:optimserver-sse-client:3.3.0'
public class SampleJavaSSEClient {
private static JobExecutionAsyncApi createSseClient() {
HttpHeadersProvider keycloakHeaders = new KeycloakHttpHeadersProvider(
KEYCLOAK_URL,
KEYCLOAK_REALM,
KEYCLOAK_CLIENT,
KEYCLOAK_USER,
KEYCLOAK_PASSWORD
);
ApiClientConfiguration apiConf = ApiClientConfiguration.builder()
.baseUri(new URI("http://" + DBOS_MASTER_HOST + ":" + DBOS_MASTER_PORT).normalize())
.requestHeaders(keycloakHeaders)
.build();
return new SseJobExecutionAsyncImpl(
new SseEventSourceFactory(apiConf),
new ObjectMapper()
);
}
public static void main() {
try {
String jobId = "created job Id";
final JobExecutionAsyncApi jobExecutionAsyncApi = createSseClient();
JobSubscriptionFilter filter = new JobSubscriptionFilter(jobId);
JobEventSource eventSource = jobExecutionAsyncApi.getJobEventSource(filter);
eventSource.statusEvents()
.subscribe(new ConsumerSubscriber<>(
jobStatusEvent -> LOGGER.info(String.format("[STATUS]: %s", jobStatusEvent.getStatus().name()))
));
eventSource.connect();
} catch (ApiException e) {
logApiException(e);
} catch (Exception e) {
LOGGER.error("An error occurred: ", e);
}
}
}
Worker (Java)
Break changes: adjust code (see this page for more details)
The 'File' type
The File ExecutionContext::getBucketFile
method has moved to InputStream ExecutionContext::getBucketFile
.
The changes to integrate are the same as for
described above
ExecutionContext::startJobAndWaitForCompletion
The signature has changed:
- Optional<SolutionDTO> startJobAndWaitForCompletion(JobSubmitRequestDTO jobSubmitRequestDTO)
+ Optional<JobSolution> startJobAndWaitForCompletion(JobSubmitRequestDTO jobSubmitRequestDTO)
The path to get the bucket reference of the solution moves from:
SolutionDTO.getSolution.getBucketValue("output-key").getBucketId
to:
JobSolution.getOutputs[name="output-key"].getBucketId
Since the worker library relies on the Master API Spring Boot starter it must comply with its configuration.
The properties in the application.yaml
file:
master.url
master.jwtKey
have moved to:
optim-server.url
optim-server.jwt.jwtKey
The former values are still taken into account to ensure backward compatibility.
The properties are migrated by calling the Python helper script :
// Might be 'python' instead of 'python3' on Windows systems
python3 -m optimserver.workerapp.migration.migrate_workerapp_config --src=3.2.2 --dst=3.3.0 --type=APPLICATION_YAML --input INPUT --output OUTPUT
Make sure the output file complies with the description below.
3.2.2:
master:
url: https://master-host/
jwtKey: ...
3.3.0:
optim-server:
url: https://master-host/
jwt:
jwtKey: ...
Worker (Python)
Deprecations (see this page for more details)
The worker shell relies on the Master API Spring Boot starter as well.
The properties in the application.yaml
file:
master.url
master.jwtKey
have moved to:
optim-server.url
optim-server.jwt.jwtKey
The former values are still taken into account to ensure backward compatibility.
The properties are migrated by calling the Python helper script :
// Might be 'python' instead of 'python3' on Windows systems
python3 -m optimserver.workerapp.migration.migrate_workerapp_config --src=3.2.2 --dst=3.3.0 --type=APPLICATION_YAML --input INPUT --output OUTPUT
Make sure the output file complies with the description below.
3.2.2:
master:
url: https://master-host/
jwtKey: ...
3.3.0:
optim-server:
url: https://master-host/
jwt:
jwtKey: ...
cplex and wod charts
Deprecations (see cplex and wod for more details)
The environment variables in the values.yaml
file
MASTER_URL
MASTER_JWTKEY
have moved to:
OPTIMSERVER_URL
OPTIMSERVER_JWT_JWTKEY
The properties are migrated by calling the Python helper script :
// Might be 'python' instead of 'python3' on Windows systems
python3 -m optimserver.helm.migration.migrate_helm_values --src=3.2.2 --dst=3.3.0 --chart=DBOS_WORKER --input INPUT --output OUTPUT
Make sure the output file complies with the description of the section below.
3.2.2:
env:
- name: MASTER_URL
value: ...
- name: MASTER_JWTKEY
valueFrom: ...
3.3.0:
env:
- name: OPTIMSERVER_URL
value: ...
- name: OPTIMSERVER_JWT_JWTKEY
valueFrom: ...
The script changes the names of the variables, so it can be applied to any values.yaml
file that describes a worker deployment