Skip to content

Conversation

@zddr
Copy link
Contributor

@zddr zddr commented Aug 23, 2023

Proposed changes

Issue Number: close #xxx

  • allow put custom jar in ${DORIS_HOME}/lib/java_extensions/custom_extension such as paimon-s3-0.4.0-incubating.jar
  • add some note for paimon and fqdn

Further comments

If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...

@github-actions
Copy link
Contributor

sh-checker report

To get the full details, please check in the job output.

shellcheck errors

'shellcheck ' returned error 1 finding the following syntactical issues:

----------

In bin/start_be.sh line 96:
if [ -d "$custom_extension_folder" ]; then
   ^-- SC2292 (style): Prefer [[ ]] over [ ] for tests in Bash/Ksh.
         ^----------------------^ SC2250 (style): Prefer putting braces around variable references even when not strictly required.

Did you mean: 
if [[ -d "${custom_extension_folder}" ]]; then


In bin/start_be.sh line 97:
  for file in "$custom_extension_folder"/*.jar ; do
               ^----------------------^ SC2250 (style): Prefer putting braces around variable references even when not strictly required.

Did you mean: 
  for file in "${custom_extension_folder}"/*.jar ; do

For more information:
  https://www.shellcheck.net/wiki/SC2250 -- Prefer putting braces around vari...
  https://www.shellcheck.net/wiki/SC2292 -- Prefer [[ ]] over [ ] for tests i...
----------

You can address the above issues in one of three ways:
1. Manually correct the issue in the offending shell script;
2. Disable specific issues by adding the comment:
  # shellcheck disable=NNNN
above the line that contains the issue, where NNNN is the error code;
3. Add '-e NNNN' to the SHELLCHECK_OPTS setting in your .yml action file.



shfmt errors

'shfmt ' returned error 1 finding the following formatting issues:

----------
--- bin/start_be.sh.orig
+++ bin/start_be.sh
@@ -94,9 +94,9 @@
 custom_extension_folder="${DORIS_HOME}/lib/java_extensions/custom_extension"
 
 if [ -d "$custom_extension_folder" ]; then
-  for file in "$custom_extension_folder"/*.jar ; do
-    export DORIS_CLASSPATH="${DORIS_CLASSPATH}:${file}"
-  done
+    for file in "$custom_extension_folder"/*.jar; do
+        export DORIS_CLASSPATH="${DORIS_CLASSPATH}:${file}"
+    done
 fi
 
 if [[ -d "${DORIS_HOME}/lib/hadoop_hdfs/" ]]; then
----------

You can reformat the above files to meet shfmt's requirements by typing:

  shfmt  -w filename


@github-actions
Copy link
Contributor

sh-checker report

To get the full details, please check in the job output.

shellcheck errors
'shellcheck ' found no issues.

shfmt errors

'shfmt ' returned error 1 finding the following formatting issues:

----------
--- bin/start_be.sh.orig
+++ bin/start_be.sh
@@ -94,9 +94,9 @@
 custom_extension_folder="${DORIS_HOME}/lib/java_extensions/custom_extension"
 
 if [[ -d "${custom_extension_folder}" ]]; then
-  for file in "${custom_extension_folder}"/*.jar ; do
-    export DORIS_CLASSPATH="${DORIS_CLASSPATH}:${file}"
-  done
+    for file in "${custom_extension_folder}"/*.jar; do
+        export DORIS_CLASSPATH="${DORIS_CLASSPATH}:${file}"
+    done
 fi
 
 if [[ -d "${DORIS_HOME}/lib/hadoop_hdfs/" ]]; then
----------

You can reformat the above files to meet shfmt's requirements by typing:

  shfmt  -w filename


@github-actions
Copy link
Contributor

sh-checker report

To get the full details, please check in the job output.

shellcheck errors
'shellcheck ' found no issues.

shfmt errors

'shfmt ' returned error 1 finding the following formatting issues:

----------
--- bin/start_be.sh.orig
+++ bin/start_be.sh
@@ -94,7 +94,7 @@
 custom_extension_folder="${DORIS_HOME}/lib/java_extensions/custom_extension"
 
 if [[ -d "${custom_extension_folder}" ]]; then
-    for file in "${custom_extension_folder}"/*.jar ; do
+    for file in "${custom_extension_folder}"/*.jar; do
         export DORIS_CLASSPATH="${DORIS_CLASSPATH}:${file}"
     done
 fi
----------

You can reformat the above files to meet shfmt's requirements by typing:

  shfmt  -w filename


@zddr
Copy link
Contributor Author

zddr commented Aug 23, 2023

run buildall

@zddr
Copy link
Contributor Author

zddr commented Aug 24, 2023

run buildall

@zddr
Copy link
Contributor Author

zddr commented Aug 24, 2023

run buildall

Copy link
Contributor

@morningman morningman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions
Copy link
Contributor

PR approved by at least one committer and no changes requested.

@github-actions github-actions bot added approved Indicates a PR has been approved by one committer. reviewed labels Aug 24, 2023
@github-actions
Copy link
Contributor

PR approved by anyone and no changes requested.

Copy link
Contributor

@morningman morningman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@morningman
Copy link
Contributor

run buildall

Copy link
Contributor

@kaka11chen kaka11chen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@hello-stephen
Copy link
Contributor

(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 46.81 seconds
stream load tsv: 541 seconds loaded 74807831229 Bytes, about 131 MB/s
stream load json: 20 seconds loaded 2358488459 Bytes, about 112 MB/s
stream load orc: 65 seconds loaded 1101869774 Bytes, about 16 MB/s
stream load parquet: 31 seconds loaded 861443392 Bytes, about 26 MB/s
insert into select: 29.3 seconds inserted 10000000 Rows, about 341K ops/s
storage size: 17161976619 Bytes

@morningman morningman merged commit ffadf09 into apache:master Aug 25, 2023
xiaokang pushed a commit that referenced this pull request Aug 25, 2023
- allow put custom jar in `${DORIS_HOME}/lib/java_extensions/custom_extension` such as `paimon-s3-0.4.0-incubating.jar`
- add some note for paimon and fqdn
@xiaokang xiaokang mentioned this pull request Aug 30, 2023
@zddr zddr deleted the add_extend_jar branch March 28, 2024 02:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by one committer. dev/2.0.1-merged reviewed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants