What's new and changed in Data Refinery
Data Refinery updates can include new features, bug fixes, and security updates. Updates are listed in reverse chronological order so that the latest release is at the beginning of the topic.
You can see a list of the new features for the platform and all of the services at What's new in IBM Cloud Pak for Data?
Installing or upgrading Data Refinery
Ready to install or upgrade Data Refinery?
- Related documentation:
Cloud Pak for Data Version 4.5.3
A new version of Data Refinery was released in October 2022 with Cloud Pak for Data 4.5.3.
Operand version: 4.5.3
This release includes the following changes:
- Security fixes
-
CVE-2022-30580, CVE-2022-25168, CVE-2022-1927, CVE-2022-1897, CVE-2022-1785, CVE-2022-1586
CVE-2019-20916
Cloud Pak for Data Version 4.5.2
A new version of Data Refinery was released in August 2022 with Cloud Pak for Data 4.5.2.
Operand version: 4.5.2
This release includes the following changes:
- Customer-reported issues fixed in this release
- DT180666: HTTPS protocol and ports 443 on deployments error
- Security fixes
-
This release includes fixes for the following security issues:
CVE-2016-5131, CVE-2016-4658
CVE-2017-9782, CVE-2017-9050, CVE-2017-9049, CVE-2017-9048, CVE-2017-9047, CVE-2017-7375, CVE-2017-6519, CVE-2017-5505, CVE-2017-5504, CVE-2017-5503, CVE-2017-5499, CVE-2017-18258, CVE-2017-17095, CVE-2017-15713, CVE-2017-15412, CVE-2017-14501, CVE-2017-14166, CVE-2017-13745, CVE-2017-0663
CVE-2018-9252, CVE-2018-9055, CVE-2018-8009, CVE-2018-6872, CVE-2018-5360, CVE-2018-20846, CVE-2018-20839, CVE-2018-20786, CVE-2018-20673, CVE-2018-20671, CVE-2018-20657, CVE-2018-20651, CVE-2018-20623, CVE-2018-20622, CVE-2018-20570, CVE-2018-20406, CVE-2018-20002, CVE-2018-19932, CVE-2018-19876, CVE-2018-19543, CVE-2018-19542, CVE-2018-19541, CVE-2018-19540, CVE-2018-19539, CVE-2018-19478, CVE-2018-19211, CVE-2018-19210, CVE-2018-19139, CVE-2018-18873, CVE-2018-18701, CVE-2018-18700, CVE-2018-18607, CVE-2018-18606, CVE-2018-18605, CVE-2018-18484, CVE-2018-18483, CVE-2018-18309, CVE-2018-18064, CVE-2018-18020, CVE-2018-17985, CVE-2018-17794, CVE-2018-17360, CVE-2018-17101, CVE-2018-16435, CVE-2018-16428, CVE-2018-16335, CVE-2018-15919, CVE-2018-15209, CVE-2018-1302, CVE-2018-12934, CVE-2018-12699, CVE-2018-12698, CVE-2018-12697, CVE-2018-12641, CVE-2018-1121, CVE-2018-10801, CVE-2018-10779, CVE-2018-1000880, CVE-2018-1000879, CVE-2018-1000876, CVE-2018-1000654, CVE-2018-1000021
CVE-2019-9937, CVE-2019-9936, CVE-2019-9923, CVE-2019-9674, CVE-2019-9543, CVE-2019-9077, CVE-2019-9075, CVE-2019-9074, CVE-2019-9071, CVE-2019-8906, CVE-2019-8905, CVE-2019-6462, CVE-2019-6461, CVE-2019-6128, CVE-2019-6110, CVE-2019-3862, CVE-2019-3861, CVE-2019-3860, CVE-2019-3859, CVE-2019-3858, CVE-2019-25059, CVE-2019-2201, CVE-2019-19244, CVE-2019-17567, CVE-2019-17543, CVE-2019-17498, CVE-2019-16866, CVE-2019-16777, CVE-2019-16775, CVE-2019-14274, CVE-2019-14250, CVE-2019-13115, CVE-2019-12972, CVE-2019-12900, CVE-2019-11026
CVE-2020-9492, CVE-2020-8492, CVE-2020-7608, CVE-2020-35512, CVE-2020-35507, CVE-2020-35496, CVE-2020-35495, CVE-2020-35494, CVE-2020-35493, CVE-2020-27748, CVE-2020-21674, CVE-2020-17049, CVE-2020-16156, CVE-2020-15095, CVE-2020-14422, CVE-2020-13790, CVE-2020-12413, CVE-2020-12401
CVE-2021-46822, CVE-2021-46195, CVE-2021-45949, CVE-2021-45944, CVE-2021-45941, CVE-2021-45940, CVE-2021-45078, CVE-2021-44568, CVE-2021-43618, CVE-2021-4217, CVE-2021-4209, CVE-2021-4166, CVE-2021-4048, CVE-2021-40330, CVE-2021-3997, CVE-2021-3974, CVE-2021-39537, CVE-2021-3927, CVE-2021-38593, CVE-2021-37404, CVE-2021-36978, CVE-2021-3618, CVE-2021-35939, CVE-2021-35938, CVE-2021-35937, CVE-2021-3468, CVE-2021-3467, CVE-2021-3443, CVE-2021-3114, CVE-2021-25317, CVE-2021-24032, CVE-2021-22570, CVE-2021-21300, CVE-2021-20193
CVE-2022-34903, CVE-2022-33980, CVE-2022-33099, CVE-2022-33068, CVE-2022-32208, CVE-2022-32206, CVE-2022-31030, CVE-2022-30556, CVE-2022-30522, CVE-2022-29404, CVE-2022-28506, CVE-2022-27943, CVE-2022-27406, CVE-2022-27405, CVE-2022-27404, CVE-2022-27337, CVE-2022-26612, CVE-2022-25310, CVE-2022-25309, CVE-2022-25308, CVE-2022-25255, CVE-2022-24765, CVE-2022-23943, CVE-2022-23648, CVE-2022-2344, CVE-2022-2343, CVE-2022-2289, CVE-2022-2288, CVE-2022-2287, CVE-2022-2286, CVE-2022-2285, CVE-2022-22844, CVE-2022-2284, CVE-2022-22719, CVE-2022-2257, CVE-2022-2231, CVE-2022-2210, CVE-2022-2208, CVE-2022-2207, CVE-2022-2206, CVE-2022-2183, CVE-2022-2182, CVE-2022-2175, CVE-2022-2129, CVE-2022-2126, CVE-2022-2125, CVE-2022-2124, CVE-2022-2058, CVE-2022-2057, CVE-2022-2056, CVE-2022-1720, CVE-2022-1619, CVE-2022-1355, CVE-2022-1354, CVE-2022-1304, CVE-2022-1122, CVE-2022-0924, CVE-2022-0909, CVE-2022-0908, CVE-2022-0891, CVE-2022-0865, CVE-2022-0562, CVE-2022-0561, CVE-2022-0530, CVE-2022-0529, CVE-2022-0391, CVE-2022-0351, CVE-2022-0235
Cloud Pak for Data Version 4.5.1
A new version of Data Refinery was released in July 2022 with Cloud Pak for Data 4.5.1.
Operand version: 4.5.1
This release includes the following changes:
- New features
-
The 4.5.1 release of Data Refinery includes the following features and updates:
- The Default Spark 3.0 & R 3.6 environment is discontinued
- If you have any Data Refinery flow jobs
set up with the Default Spark 3.0 & R 3.6
environment or a custom environment that uses Spark 3.0, the jobs will fail. Change the environment
to one of the following environments:
- Default Spark 3.2 & R 3.6
- Default Data Refinery XS
- A custom environment that does not use Spark 3.0
For information, see Data Refinery environments.
- Security fixes
-
This release includes fixes for the following security issues:
CVE-2021-40528, CVE-2021-44569, CVE-2021-44570, CVE-2021-44571, CVE-2021-44573, CVE-2021-44574, CVE-2021-44575, CVE-2021-44576, CVE-2021-44577
CVE-2022-1271, CVE-2022-1621, CVE-2022-1629, CVE-2022-22576, CVE-2022-25313, CVE-2022-25314, CVE-2022-27774, CVE-2022-27776, CVE-2022-27782, CVE-2022-29824
Cloud Pak for Data Version 4.5.0
A new version of Data Refinery was released in June 2022 with Cloud Pak for Data 4.5.0.
Operand version: 4.5.0
This release includes the following changes:
- New features
-
The 4.5.0 release of Data Refinery includes the following features and updates:
- Review and change the data types assigned in the first step of the flow
- Data Refinery automatically inserts a
first step in the flow to convert any non-string data types that it finds in the data to inferred
types. Now you can view these inferred data types and change them, if appropriate.
For information about the Convert column type operation, see GUI operations.
- View your Data Refinery data in a CSV file without running a Data Refinery flow job
- You can now export the data at the current step in your Data Refinery flow to a CSV file without saving or running a Data Refinery flow job. With this enhancement, you can now quickly save and view data that is in progress. For information, see Managing Data Refinery flows.
- Control the placement of a new column in a Data Refinery flow
- When you use an operation that can create a new column in the Data Refinery flow and you select Create
a new column for results, you can now optionally place the new column to the right of
the original column.
This option is available for the following operations:
- Calculate
- Conditional replace
- Convert column type
- Convert column value to missing
- Extract date or time value
- Math
- Replace missing values
- Replace substring
- Text
- Tokenize
- New step options give you more control of your Data Refinery flow
- Data Refinery introduces new options
for the steps that give you greater flexibility and control of the Data Refinery flow:
- Duplicate
- Insert step before
- Insert step after
You can access these options from the Steps pane. For information about the actions you can do with steps, see Managing Data Refinery flows.
- New Spark 3.2 environment for running Data Refinery flow jobs
- You can now select Default Spark 3.2 & R 3.6 when you select an
environment for a Data Refinery flow job.
The environment includes enhancements from Spark.
The Default Spark 3.0 & R 3.6 environment is deprecated.
For details, see Data Refinery environments.
- Bug fixes
-
This release includes the following fixes:
- Issue: Cannot run a Data Refinery flow job with certain unsigned data
types.
Resolution: This issue is fixed.
- Issue: Cannot view visualization charts in Data Refinery after upgrade.
Resolution: This issue is fixed.
- Issue: Cannot run a Data Refinery flow job with certain unsigned data
types.
- Security fixes
-
This release includes fixes for the following security issues:
CVE-2001-1268, CVE-2001-1269
CVE-2005-0602
CVE-2016-10739
CVE-2017-12837, CVE-2017-12883, CVE-2017-5932
CVE-2018-1000876, CVE-2018-10372, CVE-2018-10373, CVE-2018-10534, CVE-2018-10535, CVE-2018-12015, CVE-2018-12641, CVE-2018-12697, CVE-2018-12698, CVE-2018-12699, CVE-2018-12934, CVE-2018-13033, CVE-2018-14404, CVE-2018-18311, CVE-2018-18312, CVE-2018-18313, CVE-2018-18314, CVE-2018-19591, CVE-2018-19931, CVE-2018-19932, CVE-2018-20482, CVE-2018-20671, CVE-2018-20796, CVE-2018-25032, CVE-2018-6543, CVE-2018-6759, CVE-2018-6797, CVE-2018-6798, CVE-2018-6872, CVE-2018-6913, CVE-2018-7208, CVE-2018-7568, CVE-2018-7569, CVE-2018-7570, CVE-2018-7642, CVE-2018-7643, CVE-2018-8945, CVE-2018-9138, CVE-2018-9996
CVE-2019-1010204, CVE-2019-11236, CVE-2019-18276, CVE-2019-19126, CVE-2019-19956, CVE-2019-25013, CVE-2019-6488, CVE-2019-7309, CVE-2019-9169, CVE-2019-9192, CVE-2019-9923
CVE-2020-10029, CVE-2020-10543, CVE-2020-10878, CVE-2020-12723, CVE-2020-15366, CVE-2020-1751, CVE-2020-1752, CVE-2020-19131, CVE-2020-27618, CVE-2020-35492, CVE-2020-35493, CVE-2020-35494, CVE-2020-35495, CVE-2020-35496, CVE-2020-35507, CVE-2020-6096
CVE-2021-20193, CVE-2021-20197, CVE-2021-23222, CVE-2021-31879, CVE-2021-3326, CVE-2021-33503, CVE-2021-3487, CVE-2021-3517, CVE-2021-3518, CVE-2021-3537, CVE-2021-3541, CVE-2021-3580, CVE-2021-35942, CVE-2021-3634, CVE-2021-3711, CVE-2021-3712, CVE-2021-37322, CVE-2021-3800, CVE-2021-38604, CVE-2021-4160, CVE-2021-41617, CVE-2021-4189, CVE-2021-42694, CVE-2021-43818, CVE-2021-45078, CVE-2021-45960, CVE-2021-46143
CVE-2022-0778, CVE-2022-1616, CVE-2022-1620, CVE-2022-1622, CVE-2022-1623, CVE-2022-22822, CVE-2022-22823, CVE-2022-22824, CVE-2022-22825, CVE-2022-22826, CVE-2022-22827, CVE-2022-23218, CVE-2022-23219, CVE-2022-23308, CVE-2022-24757, CVE-2022-26280, CVE-2022-29155