Release Date: February 2025
What’s New?
Authentication Integration
Third-Party Authentication
We have improved our platform authentication, which is able to support third-party authentication such as Microsoft AD FS and Azure.
Add Access Token Setting
In this version, we have added a personal token settings page for each user, allowing you to create your own token settings, such as expiration time (lifetime), active or inactive status, multiple personal tokens, and the ability to delete your access token.
Data Ingestion
Support Create Delta Table on Data Catalog
In this version, Create Schema option allows users to create a schema for Delta asset, which can setup the compaction and vacuum service for Delta table.
Import & Aggregate to Existing Table
Currently, we allow the user to create the table schema and import the data later. We have developed the new feature for data importing, this feature allows users to import the data into the existing table, which is not related to import services(Table update, Stream, and Aggregate), service chain, and Workflow service. There are 3 upload methods for import with one time including One Time (will have another option for Overwrite Source), Schedule, and Stream.
Limitation: 1.) No Support multiple Import Services. If a user creates the import service or aggregate service after using import/aggregate to existing table, the import service cannot run as usual. 2.)If a state table with an encryption table setting was created before version 4.6.0, it cannot be used for importing into an existing table in append mode.
Users need to have ‘Edit’ Permission of the selected data asset.
Apply State Table To All Time-series Table
Previously, this state table setting was able to do for streaming import. We have enhanced the state table option for others import types such as schedule and one time uploading. With this ability, users can create the data tiering from state table by selecting your own storage location of each state.
Limitation: 1.)This is capable for Time-series data only. 2.) State table cannot change the source type(Data Asset Type). 3.)State table cannot change schema, time-domain, partition, and encryption.
Databricks Ingestion
As Databricks is well-known in the big data industry and widely used worldwide, we have implemented a feature to support data from Databricks, allowing users to ingest data into our platform for processing and analytics.
Data Management
Compute Task Monitoring
Our platform has developed the feature to display your compute tasks monitoring, which is the Zeus(Spark jobs) via the Blendata user interface. User can kill your own task jobs such as export or data query task and can see the jobs under Zeus that you have an access permission. You can see your task queue and progress status as well. This feature excludes Zeppelin jobs in this version.
Data Lineage for Workflow
We have added a new menu for Workflow in the Data Lineage menu. User can see the streamline of Workflow and can access to your workflow from this menu. The Workflow upstream will display when users already used the SQL service.
System Table Retention with Minimum 90 days
We have enhanced the system table retention feature to minimize the retention period to 90 days. This ensures that old records in the data asset are retained for 90 days or longer before the system automatically removes them. This functionality aligns with the NECTEC standard policy.
Service Monitoring
This new menu is used to monitor all services in the Blendata platform, a user is able to filter a period with minutes or hours to see the job running overview in the chart and job number and see the service job name that is running currently or in the queue. Moreover, users can see the service lists of each latest result status like Error, Incomplete, and Skipped to see the log or job details.
This feature needs to be enabled in the role or user management first to see this menu.
Distribute Data Assets Across Multiple Database Schemas
Enhanced data asset organization by allowing assets to be organized by role or user, helping to categorize data assets more effectively. This new feature reduces the time required for administrators to assign access to individual assets for each user.
Asset Owner Transfer
This feature has been improved to allow the transfer of all assets from all Realms, including data assets, services, service chains, and workflow assets, or some assets to another person via the User Management menu.
Administrator role can transfer the asset owner only.
Data Processing and Analytics
BDE Assistant for SQL Generator
Blendata Assistant simplifies tasks with its initial function, the ‘SQL Generator.’ Seamlessly integrated into the Query panel, it offers support when you’re stuck with code. The interface provides a text input where users can describe in natural language what they wish to explore. Blendata Assistant then translates these requirements into SQL queries.
Add Service Type Import Service in the Workflow
We have added a new service type option in the Workflow: import service job. This allows you to import a new data asset or data into existing assets through the Workflow Management menu.
Import Library for Notebook environment
Platform allows users to import libraries directly into Notebook environment. Users need to upload or move the library file into an environment(python_scripts folder) and run the paragraph ‘import file name’ in the Notebook.
Support interpreter python(%python) only.
Specific Default Zeus by Role and User
Specific Zeus for each role to prevent resource contention and manage resources separately for each role and allocate Zeus usage appropriately based on each role.
ERROR ID in Log
Added the error ID that indicates the error module in the log file, which can help to identify the error easily.
Others
Auto Migration when Upgrade Version
We have improved the initial database setup of platform using a script-based approach to enhance flexibility in database management for future versions. This also simplifies integration with container-based environments.
Limitation: Users need to install version to be V.4.5.x or upgrade version to be V.4.5.x first.
Fixed Bug:
My Profile and Data Catalog
- BDEN-605 Fixed the issue with page numbers in the My Profile menu, which displayed incorrectly when switching view styles.
- BDEN-621 Fixed the error where users could not search the number in tag field.
- BDEN-786 Fixed an issue where logging in with a different account would retrieve the permissions of the previously logged-in account.
- BDEN-1387 Fixed the bug that users could not add a new tag, if tag’s name was a part of an existing tag
- BDEN-1726 Resolved the loading performance problem on Data Catalog menu.
- BDEN-1837 Solved the issue where view permission allowed copying the JSON schema and accessing all data source options.
- BDEN-1965 Fixed a data lineage bug where system jobs did not appear in the service tab.
- BDEN-2012 Fixed the Delta compact service issue.
- BDEN-2088 The default picture of user account was missed, so this issue need to be fixed.
- BDEN-3580 Fixed a notification icon error where reading a notification should change the text to a soft color instead of deleting the notification.
- BDEN-4000 Revise the arrow inside schema selection dropdown, which is faded when using dark theme.
- BDEN-4021 Updated the Blendata website link in the product information popup.
- BDEN-4195 Resolved the issue where users without permission could not view the published asset.
- BDEN-4136 Fixed the issue where the size statistics display of tables in a customized schema appeared incorrect.
- BDEN-4232 Fixed an incorrect pagination display.
- BDEN-4338 Fixed a data lineage error where the unlinked system job was still displayed in downstream section.
- BDEN-4683 Resolved the issue where user could not edit the profile information in Profile page.
- BDEN-5059 Resolved the Data Lineage error where the Notebook did not display the environment on Downstream after adding the data asset.
- BDEN-5094 Fixed the incorrectly status of services on Data Lineage menu.
- BDEN-5137 Resolved the login issue where switching between Realms without Zeus prevented the user from logging in.
Import Data
- BDEN-1801 Fixed the error where some shape files could not be imported into the system.
- BDEN-1974 Fixed the error of incomplete storage path options when users enabled a Service State table.
- BDEN-2024 Resolved a bug where could not select ‘Upload Method’ during RDBMS importing.
- BDEN-2025 Resolved an issue in import settings where the system would freeze if users configured the Setting Output Options, returned to the Customize Field step, and proceeded to the next step.
- BDEN-2074 Fixed error when users enabled a service for delta table, but the service are not appeared.
- BDEN-2075 Fixed error when users enabled a service for delta table and setup for change customization, then the service was changed to disable.
- BDEN-2112 Fixed a failure error during file path import when switching from Stream to Schedule run service.
- BDEN-2113 Fixed a bug where state recovery and refresh jobs were not appeared during file path import setting.
- BDEN-2117 Fixed an error notification that appeared when the table name was blank, but users clicked Next to proceed to the next step for data importing.
- BDEN-2150 Fixed a bug where the ‘Move to Trash After Finishing Import Data’ option was missing when users imported data via the Internal File Path.
- BDEN-2156 Resolved the file path importing issue where the ‘Execute Immediately’ option was set to disabled, but switching to other tabs and returning to the current tab caused the option to appear as enabled.
- BDEN-2339 Fixed an error occurring when users created a table schema with an internal table.
- BDEN-2257 Fixed an incorrect imported record value when creating a data table with a partition that is not of the date type.
- BDEN-2338 Resolved the data importing issue where data could not be imported via the internal file path when encrypting data, setting a state table with a schedule, and using overwrite mode.
- BDEN-4165 Fixed an issue where a new account could not create a new data asset on the platform immediately.
- BDEN-4358 Fixed the Cassandra importing, selected a table, and set a setting. Then, the platform displayed ‘Error 600: Failed to open a native connection to Cassandra’.
- BDEN-4681 Resolved the issue with data import where no results were displayed when previewing the table or querying on the Data Exploration page.
- BDEN-4778 Resolved the importing issue where uploading new data with mismatched column names caused changes to the path and sink mode values in Data Source Options.
- BDEN-4829 Resolved the CDC importing issue where the custom state menu was missing in the Stream Settings menu.
- BDEN-5008 Fixed an issue where source files with a ‘+’ character in the filename could not be deleted during the import file process.
- BDEN-5023 Fixed an import data via internal file path issue where the source file was moved to trash even if the user disabled the move to trash option.
- BDEN-5024 Resolved the issue that imported Internal Table(Stream upload) and this data asset could not show data on Partition Management menu.
- BDEN-5025 Fixed the ‘Error 600: Failed to open native connection to Cassandra’ bug during connected to Cassandra.
- BDEN-5088 Fixed the blank page error when users import table from Spark Internal Meta-store.
Data Processing
- BDEN-771 Resolved the issue where the menu would freeze when importing JSON files with numerous records or performing data processing in the Data Preparation menu and other related menus.
- BDEN-1175 Fixed the issue with Zeus and pool settings, where users configured Zeus and pool for query commands and replaced the query in the tab, but the data processing did not use the configured Zeus and pool settings.
- BDEN-1184 Fixed the bug where long names (Zeus and pool names) in ‘Operating On’ overlapped with the preview result box in Data Exploration.
- BDEN-1281 Fixed the error where the run name and task name did not the full name when they were shortened.
- BDEN-1315 Fixed the dialog size issue that occurred when users zoomed in or out in an internet browser.
- BDEN-1396 Fixed the issue where the saved query name has been truncated during editing due to the name character limitation.
- BDEN-1423 Fixed the bug that platform displayed an encrypt data icon but the user did not select the table as the Encrypt data during aggregate table creation.
- BDEN-1644 Resolved the SQL Editor error where users could not execute SQL queries with uppercase table names.
- BDEN-2202 Resolved the error when creating an aggregate table with a Delta table and selecting ‘Enable service for Delta table,’ but the Delta services were not created.
- BDEN-2373 Fixed the error where users could not save an aggregate table into an existing table.
- BDEN-3440 Fixed a history notification list error when users used CAST() function in SQL command.
- BDEN-3494 Resolved an issue in Notebook where CPU usage was displayed incorrectly when running Spark with Dynamic Allocation.
- BDEN-3584 Could not put the uppercase text in the result column filter.
- BDEN-3606 Revised the action message to start with an uppercase letter.
- BDEN-4007 Updated the text color to improve visibility in dark theme.
- BDEN-4206 Fixed the issue with aggregates using Delta tables where the platform sent path information with a null value to the backend.
- BDEN-4342 Fixed the permission error where users were set at the manage level, but users performed at the view level.
- BDEN-4415 Fixed the encrypt table, which did not show the encrypt icon in the source type.
- BDEN-4436 Fixed an error where users could not select the time range in query details.[Simple Query]
- BDEN-4565 Fixed the incorrect information on CPU usage in the Notebook menu.
- BDEN-4600 Resolved the issue where the long-lived token for Notebook expired.
- BDEN-4611 Resolved the issue where Notebooks not created within the Enterprise or not stored in the Enterprise database failed to process.
Service Management
- BDEN-682 Resolved the Alert service error that occurred due to incorrect SQL syntax and changed the error message to be ‘SQL Syntax Error’.
- BDEN-720 Fixed the bug where text exceeded the boundaries of the service parameters box in the Edit Batch Service settings.
- BDEN-982 Fixed the bug in the Alert service that allowed users to leave the file name blank and platform did not limit the file name text.
- BDEN-1557 Fixed the bug where the cron schedule value box was displayed incorrectly.
- BDEN-1643 Resolved the issue where the user interface did not follow the design.
- BDEN-1872 Fixed an issue where services could not utilize other Zeuses to execute.
- BDEN-1875 Fixed an issue where services could not utilize other Zeuses to create a service (job triggering immediately).
- BDEN-2032 Solved the issue where the service time configuration did not update according to the specified time settings.
- BDEN-2076 Fixed a system service error where users forcibly stopped a process, causing the system to display ‘Error 404: (undefined).
- BDEN-3833 Fixed the ‘Add Partition Job’ error where the job trigger changed to ‘Specific Time,’ causing the date to display an invalid value.
- BDEN-3980 Resolved the loading performance issue that caused ‘Error 600’ on some service pages (Stream and Aggregate).
- BDEN-3976 Fixed an issue where the Delta compact service job displayed an error.
- BDEN-3979 Disable ‘Delete Service’ option of the first service chain job.
- BDEN-4164 Fixed an error where other accounts could not run the service chains.
- BDEN-4389 Fixed the white background error during auto refresh in the dark theme.
- BDEN-4414 Fixed a compression invalid type issue when setting up state table service.
- BDEN-4426 Fixed the export job did not show the tag name.
- BDEN-4475 Fixed the bug where editing the Alert Scheduler with email and could not click ‘Save’.
- BDEN-4490 Resolved the tag error where users created a tag and saved it immediately without waiting for the tag to appear.
- BDEN-4801 Fixed a invalid permission in Service Chains menu.
Workflow Management
- BDEN-1385 Fixed an issue where users were able to enter negative values in the Retry at Most and Timeout Duration fields.
- BDEN-1537 Resolved a workflow that did not remove an upstream lineage from edited or deleted chain job.
- BDEN-2176 Fixed the error design of dark theme.
- BDEN-1728 Fixed the notification email error where the system did not send notifications when users set an alert scheduler.
- BDEN-3446 Resolved the issue where users with Manager/Owner permission could not edit the workflow version information.
- BDEN-4117 Resolved the loading issue when selecting the view error log of an attempt.
- BDEN-4139 Added a loading popup when changing a task’s attempts to align with the view log menu in the history tab.
- BDEN-4043 Fixed the notification issue where an email was not sent when workflow deployment failed.
- BDEN-4092 Fixed the notification error where an email was not sent when a workflow was shared with a user or role.
- BDEN-4114 Fixed a workflow creation error where creating a new workflow with a retry policy caused the save button to be disabled.
- BDEN-4118 Updated the ‘QUEUE’ status color in dark theme to enhance visibility.
Visualization and Dashboard
- BDEN-919 Fixed the bug where the global filter could not use AND and OR as a value filter.
- BDEN-1568 Resolved the error where there was a comma in the year column (string) in the line chart.
- BDEN-1725 Fixed the bug where map drill down returning an incorrect latitude and longitude value.
- BDEN-1729 Fixed the bug that visualizations with dynamic viz could not retain local filter after applied global filter or refreshed dashboard.
- BDEN-1731 Fixed the error where the platform got stuck on loading when users imported a JSON visualization file.
- BDEN-1833 Fixed the error where user could not create a dashboard.
- BDEN-3451 Fixed the dashboard error that displayed incorrect date format in global filters.
- BDEN-3893 Resolved the publish dashboard issue that displayed ‘Error 600’ and showed ‘No Data Available’ in the filter list.
- BDEN-3969 Fixed a bug where used specify own function and applied ‘In’ or ‘Not In’, the result displayed incorrectly.
- BDEN-3974 Fixed a visualization error where annotation types with boxes displayed incorrect colors.
- BDEN-3983 Fixed the issue where the leaflet map with the dark theme displayed incorrect text color in the popup template.
- BDEN-3996 Revised the menu name of ‘Remove From Favorite’.
- BDEN-4200 Fixed the freezing issue during application export when the application contained a blank visualization.
- BDEN-4224 Fixed the bug where the ‘Apply’ button exceeded the edge of the date when setting the end of the range with a specific value in the global filter setting.
- BDEN-4557 Fixed the publish visualization error where it did not display the visualization data.
- BDEN-4886 Resolved the error where users applied a filter using a Combo box with an SQL query.
Setting
- BDEN-844 Fixed the error that occurred when users either did not enter the realm or entered an incorrect realm, which caused the Zeus name to disappear from the list.
- BDEN-1055 Resolved the issue where the role components in the Import menu displayed incorrect Operations permissions.
- BDEN-1117 Resolved the slow loading performance for temporary compute.
- BDEN-1128 Resolved the slow loading performance for compute activate and deactivate.
- BDEN-3472 Fixed the Realm management issue where Zeus was moved to another Realm but still appeared in the old Zeus group.
- BDEN-3663 Removed the duplicate Zeus name from the advanced option list when all roles were set to the same Zeus.
- BDEN-3952 Fixed Realm management to allow the removal of a Realm that contains Zeus.
- BDEN-1191 Fixed the wording and improved design of billing history email content.
- BDEN-1192 Fixed a mismatch in duration values displayed between the billing history and view detail(Activity Log).
- BDEN-1221 Resolved the incorrect computation of online credit usage after the billing cycle was processed.
- BDEN-2137 Fixed the error that platform did not show the details of Temporary Compute in License Management page.
- BDEN-2138 Resolved the incorrect information in License Billing History, where the computed details and the details in ‘View Detail’ did not match.
- BDEN-2194 Fixed an incorrect license detail error in an email. (License Type/Credit Limit)
- BDEN-2057 Resolved a mismatch license when used specific hardwareID configuration.
- BDEN-2201 Fixed a bug that a license displayed incorrect value of CPU usage.
- BDEN-3931 Fixed a bug in the Activity Log results where they mismatched the module filter.
- BDEN-3933 Fixed a text color issue in the dark theme where it was displayed incorrectly.
- BDEN-3470 Resolved an error where the Zeus in the service was displayed incorrectly, mismatching with the advanced Zeus during CDC import.
- BDEN-3962 Fixed the incorrect icon color of Auto Start/Stop.
- BDEN-3991 Fixed a lost information of license in dark theme.
- BDEN-3993 Fixed an issue with the duration license type where users activated the license, but the status still showed the warning ‘ACTIVATE LICENSE’.
- BDEN-4209 Fixed the error where a user account with SQL editor menu could not be deleted.
- BDEN-4361 Fixed incorrect sidebar menu for Administrator role, which needs to access only LDAP and User Management menu.
- BDEN-4369 Resolved the bug when users deleted the user account, the platform showed an error message but the accounts were deleted.
- BDEN-4366 Revised the error message that was displayed when a user did not fill in the expiration date.
- BDEN-4599 Resolved an issue where the role sidebar menu permissions were incorrect.
- BDEN-4777 Fixed the License Management error where the values for Credit Used, Available Credit, and Total Used were displayed incorrectly after the billing cycle cutoff.
- BDEN-5062 Fixed the ‘Auto start/stop configuration update fail!’ error when setup Auto Start/Stop Zeus configuration on License Management.
- BDEN-5126 Resolved the user information settings error where the updated information was not displayed after editing.
Security
- Sonarqube
- Pentest
- BDEN-4847 Fixed a remote code execution issue. (OWASP Top 10 A03:2021 – Injection)
- BDEN-4849 Fixed an arbitrary file read issue. (OWASP Top 10 A01:2021 – Broken Access Control)
- BDEN-4861 Fixed an insecure direct object references (IDOR) issue. (OWASP Top 10 A01:2021 – Broken Access Control)
- BDEN-4861 Resolved a broken access control issue. (OWASP Top 10 A01:2021 – Broken Access Control)
- BDEN-4863 Resolved a broken session management (Session Fixation) issue. (OWASP Top 10 A07:2021 – Identification and Authentication Failures)
- BDEN-4864 Resolved an unrestricted file upload issue. (OWASP Top 10 A04:2021 – Insecure Design)
- BDEN-4865 Resolved a sensitive information disclosure issue. (OWASP Top 10 A05:2021 – Security Misconfiguration)
- BDEN-4863 Fixed a broken session management (logout does not terminate the user session) issue. (OWASP Top 10 A07:2021 – Identification and Authentication Failures)
- BDEN-4863 Fixed a broken session management (long session idle timeout) issue. (OWASP Top 10 A05:2021 – Security Misconfiguration)
- Error Message Containing Stack Traces (OWASP Top 10 A05:2021 – Security Misconfiguration)
- BDEN-4890 Fixed an insecure content-security-policy header issue. (OWASP Top 10 A05:2021 – Security Misconfiguration)
- BDEN-4891 Fixed an error of JSON web token (JWT) stored in local storage. (OWASP Top 10 A05:2021 – Security Misconfiguration)
- Missing Server-Side Input Validation (OWASP Top 10 A04:2021 – Insecure Design)
- BDEN-4893 Solved an overly permissive cross origin resource sharing (CORS) policy (OWASP Top 10 A05:2021 – Security Misconfiguration)
- BDEN-4894 Resolved a password disclosure in HTTP responses issue. (OWASP Top 10 A02:2021 – Cryptographic Failures)
- BDEN-4895 Support of Weak TLS Cipher Suites. (OWASP Top 10 A02:2021 – Cryptographic Failures)
- BDEN-4896 Improved an username enumeration problem. (OWASP Top 10 A07:2021 – Identification and Authentication Failures)
Improvement:
- BDEN-775 Added tooltip options to display data for either a specific point or all data along the X-axis.
- BDEN-1170 Displayed the license ID on the website user interface.
- BDEN-1250 Created a temporary license type in the License Management.
- BDEN-143 Improved Service Chain Job feature to be able to insert jobs between any indexes.
- BDEN-1733 Enabled users to specify a date and time in the global filter on the dashboard.
- BDEN-1778 Improved system performance during partition deletion.
- BDEN-714 Improved Notebook license calculation to be an actual CPU usage.
- BDEN-2188 Changed the Credit Calculation Change CPU from integer to decimal value.
- BDEN-3605 Improved the loading performance of SQL Editor menu.
- BDEN-3623 Improved a side menu bar design.
- BDEN-3641 Enhanced the cron schedule to run the first round according to the specified conditions.
- BDEN-2663 Revised the default data preview of the View table to 24 hours and added a time range filter to the preview tab, improving the View table’s preview performance.
- BDEN-4006 Enable a user to upload file that file name contained a space.
- BDEN-4360 Improved the design of side bar menu in the platform.
