This Week in Databend #103

July 23, 2023 · 3 min read

PsiACE

Stay up to date with the latest weekly developments on Databend!

Databend is a modern cloud data warehouse, serving your massive-scale analytics needs at low cost and complexity. Open source alternative to Snowflake. Also available in the cloud: https://app.databend.com .

What's On In Databend

Stay connected with the latest news about Databend.

Creating Network Policies

Network Policies are used to manage access to Databend services. They can also be utilized to restrict access to a SQL user account based on users' IP addresses.

To create a set of network policies:

CREATE NETWORK POLICY <policy-name> ALLOWED_IP_LIST=(<allowed-ip>) BLOCKED_IP_LIST=(<blocked-ip>) COMMENT=<policy-comment>

To restrict access to a SQL user account:

CREATE USER <user-name> IDENTIFIED BY <user-password> WITH SET NETWORK POLICY=<policy-name>

--- OR ---

ALTER USER <user-name> WITH SET NETWORK POLICY=<policy-name>

If you are interested in learning more, please check out the resources listed below.

Code Corner

Discover some fascinating code snippets or projects that showcase our work or learning journey.

Profiling Table Scan

Databend now supports wait_time for profiling table scans. This feature offers insights into time spent reading data from storage, assisting in determining whether a query is I/O or CPU bound.

If you are interested in learning more, please check out the resources listed below:

PR #12158 | feat(planner): Support profile table scan

Highlights

We have also made these improvements to Databend that we hope you will find helpful:

Added intersect_count and to_yyyymmddhh functions.
Added fuse_column system function.
Read document Docs | RECLUSTER TABLE to understand the updated reclustering mechanism.
Read documents Docs | Fuse Engine and Docs | ALTER TABLE OPTION to learn the options available in Fuse Engine and how to modify them.
Read document Docs | Vacuum Dropped Tables to learn about our latest enterprise feature and how it can help you save storage space.

What's Up Next

We're always open to cutting-edge technologies and innovative ideas. You're more than welcome to join the community and bring them to Databend.

Renaming TSV file format to Text

Databend supports a file format called TSV, which may mislead users into thinking that its only difference from CSV is the field delimiter.

In reality, TSV corresponds to the text format of MySQL/Hive/PostgreSQL and uses escaping instead of quoting to handle delimiters in values (MySQL text supports quoting but not by default). ClickHouse also uses it for transferring data to/from MySQL.

Therefore, we suggest renaming TSV to TEXT.

Issue #11987 | Rename TSV file format to Text

Please let us know if you're interested in contributing to this feature, or pick up a good first issue at https://link.databend.rs/i-m-feeling-lucky to get started.