<?xml version="1.0" encoding="utf-8"?>
<feed xmlns="http://www.w3.org/2005/Atom">
    <id>https://tis.pub/blog</id>
    <title>零代码数据集成平台 + ChatBI智能问数 Blog</title>
    <updated>2026-05-29T00:00:00.000Z</updated>
    <generator>https://github.com/jpmonette/feed</generator>
    <link rel="alternate" href="https://tis.pub/blog"/>
    <subtitle>零代码数据集成平台 + ChatBI智能问数 Blog</subtitle>
    <icon>https://tis.pub/img/favicon.ico</icon>
    <entry>
        <title type="html"><![CDATA[TIS 伴生操作插件开发指南：从零上手 Manipulate 体系]]></title>
        <id>accompanying-plugin-manipulate</id>
        <link href="https://tis.pub/blog/accompanying-plugin-manipulate"/>
        <updated>2026-05-29T00:00:00.000Z</updated>
        <summary type="html"><![CDATA[本文面向刚加入 TIS 团队的开发工程师。读完本文，你将理解"伴生操作类"是什么、为什么需要它、以及如何从零开发一个完整的 Manipulate 插件。]]></summary>
        <content type="html"><![CDATA[<blockquote><p>本文面向刚加入 TIS 团队的开发工程师。读完本文，你将理解"伴生操作类"是什么、为什么需要它、以及如何从零开发一个完整的 Manipulate 插件。</p></blockquote><hr><h2 class="anchor anchorWithStickyNavbar_LWe7" id="一背景什么是伴生操作类">一、背景：什么是"伴生操作类"？<a class="hash-link" href="#一背景什么是伴生操作类" title="Direct link to heading">​</a></h2><p>TIS 的核心实体（如数据管道 <code>DataXProcessor</code>、本体域 <code>OntologyDomain</code>）本身只负责描述配置，不直接处理用户在界面上触发的"操作"——比如克隆、开启某个功能、触发推断任务等。</p><p>这些"操作"由<strong>伴生操作类（Manipulate）</strong>承担。每个核心实体可以挂载若干个 Manipulate 插件，每个插件对应一种用户可触发的操作。</p><div class="codeBlockContainer_Ckt0 theme-code-block" style="--prism-color:#393A34;--prism-background-color:#f6f8fa"><div class="codeBlockContent_biex"><pre tabindex="0" class="prism-code language-text codeBlock_bY9V thin-scrollbar"><code class="codeBlockLines_e6Vv"><span class="token-line" style="color:#393A34"><span class="token plain">核心实体（如 OntologyDomain）</span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">  └── 伴生操作类 1：EnableChatBI（开启智能问数）</span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">  └── 伴生操作类 2：InferOntologyFromLLM（LLM 推断本体关系）</span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">  └── 伴生操作类 N：...</span><br></span></code></pre><div class="buttonGroup__atx"><button type="button" aria-label="Copy code to clipboard" title="Copy" class="clean-btn"><span class="copyButtonIcons_eSgA" aria-hidden="true"><svg class="copyButtonIcon_y97N" viewBox="0 0 24 24"><path d="M19,21H8V7H19M19,5H8A2,2 0 0,0 6,7V21A2,2 0 0,0 8,23H19A2,2 0 0,0 21,21V7A2,2 0 0,0 19,5M16,1H4A2,2 0 0,0 2,3V17H4V3H16V1Z"></path></svg><svg class="copyButtonSuccessIcon_LjdS" viewBox="0 0 24 24"><path d="M21,7L9,19L3.5,13.5L4.91,12.09L9,16.17L19.59,5.59L21,7Z"></path></svg></span></button></div></div></div><p>用户在界面上点击某个操作按钮时，TIS 框架会找到对应的 Manipulate 实例，调用其 <code>manipuldateProcess</code> 方法。</p><hr><h2 class="anchor anchorWithStickyNavbar_LWe7" id="二类体系总览">二、类体系总览<a class="hash-link" href="#二类体系总览" title="Direct link to heading">​</a></h2><div class="codeBlockContainer_Ckt0 theme-code-block" style="--prism-color:#393A34;--prism-background-color:#f6f8fa"><div class="codeBlockContent_biex"><pre tabindex="0" class="prism-code language-text codeBlock_bY9V thin-scrollbar"><code class="codeBlockLines_e6Vv"><span class="token-line" style="color:#393A34"><span class="token plain">BasicManipuldateProcessor&lt;T&gt;          ← 基类，封装持久化通用逻辑（本文重点）</span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">  ├── DefaultDataXProcessorManipulate  ← DataX 管道的伴生操作基类</span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">  │     ├── BatchJobCrontab            ← 定时任务配置</span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">  │     ├── AddMonitorForEvents        ← 告警监控配置</span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">  │     └── ExportTISPipelineToDolphinscheduler  ← 导出到调度系统</span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">  └── OntologyDomainManipulate         ← 本体域的伴生操作基类</span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">        ├── EnableChatBI               ← 开启智能问数（需持久化）</span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">        └── InferOntologyFromLLM       ← LLM 推断本体关系（不持久化）</span><br></span></code></pre><div class="buttonGroup__atx"><button type="button" aria-label="Copy code to clipboard" title="Copy" class="clean-btn"><span class="copyButtonIcons_eSgA" aria-hidden="true"><svg class="copyButtonIcon_y97N" viewBox="0 0 24 24"><path d="M19,21H8V7H19M19,5H8A2,2 0 0,0 6,7V21A2,2 0 0,0 8,23H19A2,2 0 0,0 21,21V7A2,2 0 0,0 19,5M16,1H4A2,2 0 0,0 2,3V17H4V3H16V1Z"></path></svg><svg class="copyButtonSuccessIcon_LjdS" viewBox="0 0 24 24"><path d="M21,7L9,19L3.5,13.5L4.91,12.09L9,16.17L19.59,5.59L21,7Z"></path></svg></span></button></div></div></div><p><strong>关键包路径：</strong></p><table><thead><tr><th>类</th><th>路径</th></tr></thead><tbody><tr><td><code>BasicManipuldateProcessor</code></td><td><code>tis-plugin/.../plugin/manipulate/</code></td></tr><tr><td><code>DefaultDataXProcessorManipulate</code></td><td><code>tis-plugin/.../datax/</code></td></tr><tr><td><code>OntologyDomainManipulate</code></td><td><code>tis-plugin/.../plugin/ontology/</code></td></tr><tr><td><code>EnableChatBI</code> / <code>InferOntologyFromLLM</code></td><td><code>tis-ontology-plugin/.../plugin/ontology/</code></td></tr></tbody></table><hr><h2 class="anchor anchorWithStickyNavbar_LWe7" id="三核心接口与基类详解">三、核心接口与基类详解<a class="hash-link" href="#三核心接口与基类详解" title="Direct link to heading">​</a></h2><h3 class="anchor anchorWithStickyNavbar_LWe7" id="31-ipluginstoremanipuldateprocessor">3.1 <code>IPluginStore.ManipuldateProcessor</code><a class="hash-link" href="#31-ipluginstoremanipuldateprocessor" title="Direct link to heading">​</a></h3><p>这是所有 Manipulate 类必须实现的顶层接口，只有一个方法：</p><div class="language-java codeBlockContainer_Ckt0 theme-code-block" style="--prism-color:#393A34;--prism-background-color:#f6f8fa"><div class="codeBlockContent_biex"><pre tabindex="0" class="prism-code language-java codeBlock_bY9V thin-scrollbar"><code class="codeBlockLines_e6Vv"><span class="token-line" style="color:#393A34"><span class="token keyword" style="color:#00009f">interface</span><span class="token plain"> </span><span class="token class-name">ManipuldateProcessor</span><span class="token plain"> </span><span class="token punctuation" style="color:#393A34">{</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">    </span><span class="token keyword" style="color:#00009f">void</span><span class="token plain"> </span><span class="token function" style="color:#d73a49">manipuldateProcess</span><span class="token punctuation" style="color:#393A34">(</span><span class="token class-name">IPluginContext</span><span class="token plain"> pluginContext</span><span class="token punctuation" style="color:#393A34">,</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">                            </span><span class="token class-name">UploadPluginMeta</span><span class="token plain"> pluginMeta</span><span class="token punctuation" style="color:#393A34">,</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">                            </span><span class="token class-name">Optional</span><span class="token generics punctuation" style="color:#393A34">&lt;</span><span class="token generics class-name">Context</span><span class="token generics punctuation" style="color:#393A34">&gt;</span><span class="token plain"> context</span><span class="token punctuation" style="color:#393A34">)</span><span class="token punctuation" style="color:#393A34">;</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain"></span><span class="token punctuation" style="color:#393A34">}</span><br></span></code></pre><div class="buttonGroup__atx"><button type="button" aria-label="Copy code to clipboard" title="Copy" class="clean-btn"><span class="copyButtonIcons_eSgA" aria-hidden="true"><svg class="copyButtonIcon_y97N" viewBox="0 0 24 24"><path d="M19,21H8V7H19M19,5H8A2,2 0 0,0 6,7V21A2,2 0 0,0 8,23H19A2,2 0 0,0 21,21V7A2,2 0 0,0 19,5M16,1H4A2,2 0 0,0 2,3V17H4V3H16V1Z"></path></svg><svg class="copyButtonSuccessIcon_LjdS" viewBox="0 0 24 24"><path d="M21,7L9,19L3.5,13.5L4.91,12.09L9,16.17L19.59,5.59L21,7Z"></path></svg></span></button></div></div></div><p><strong>你不需要直接实现这个接口</strong>。继承 <code>BasicManipuldateProcessor</code> 后，框架已经帮你实现了。</p><hr><h3 class="anchor anchorWithStickyNavbar_LWe7" id="32-basicmanipuldateprocessort">3.2 <code>BasicManipuldateProcessor&lt;T&gt;</code><a class="hash-link" href="#32-basicmanipuldateprocessort" title="Direct link to heading">​</a></h3><p>这是所有 Manipulate 插件的基类，封装了完整的持久化流程：</p><div class="codeBlockContainer_Ckt0 theme-code-block" style="--prism-color:#393A34;--prism-background-color:#f6f8fa"><div class="codeBlockContent_biex"><pre tabindex="0" class="prism-code language-text codeBlock_bY9V thin-scrollbar"><code class="codeBlockLines_e6Vv"><span class="token-line" style="color:#393A34"><span class="token plain">用户触发操作</span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">    ↓</span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">manipuldateProcess()  ← final，不可覆盖</span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">    ↓</span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">ManipuldateUtils.instance()  ← 解析请求上下文（是新增/更新/删除？）</span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">    ↓</span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">loadPluginStore()  ← 抽象方法，子类实现，返回对应的 IPluginStore</span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">    ↓</span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">[删除] deleteFromStore()</span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">[新增/更新] 判重 → replaceInStore()</span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">    ↓</span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">afterManipuldateProcess()  ← 钩子，子类按需覆盖（如触发同步任务）</span><br></span></code></pre><div class="buttonGroup__atx"><button type="button" aria-label="Copy code to clipboard" title="Copy" class="clean-btn"><span class="copyButtonIcons_eSgA" aria-hidden="true"><svg class="copyButtonIcon_y97N" viewBox="0 0 24 24"><path d="M19,21H8V7H19M19,5H8A2,2 0 0,0 6,7V21A2,2 0 0,0 8,23H19A2,2 0 0,0 21,21V7A2,2 0 0,0 19,5M16,1H4A2,2 0 0,0 2,3V17H4V3H16V1Z"></path></svg><svg class="copyButtonSuccessIcon_LjdS" viewBox="0 0 24 24"><path d="M21,7L9,19L3.5,13.5L4.91,12.09L9,16.17L19.59,5.59L21,7Z"></path></svg></span></button></div></div></div><p><strong>你需要实现的方法只有两个：</strong></p><h4 class="anchor anchorWithStickyNavbar_LWe7" id="loadpluginstore必须实现"><code>loadPluginStore()</code>（必须实现）<a class="hash-link" href="#loadpluginstore必须实现" title="Direct link to heading">​</a></h4><p>告诉基类"把数据存到哪里"。从 <code>itemsProcessor.getPluginMeta()</code> 中提取上下文（如 domain 名、pipeline 名），返回对应的 <code>IPluginStore</code>。</p><div class="language-java codeBlockContainer_Ckt0 theme-code-block" style="--prism-color:#393A34;--prism-background-color:#f6f8fa"><div class="codeBlockContent_biex"><pre tabindex="0" class="prism-code language-java codeBlock_bY9V thin-scrollbar"><code class="codeBlockLines_e6Vv"><span class="token-line" style="color:#393A34"><span class="token annotation punctuation" style="color:#393A34">@Override</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain"></span><span class="token keyword" style="color:#00009f">protected</span><span class="token plain"> </span><span class="token class-name">IPluginStore</span><span class="token generics punctuation" style="color:#393A34">&lt;</span><span class="token generics class-name">MyManipulate</span><span class="token generics punctuation" style="color:#393A34">&gt;</span><span class="token plain"> </span><span class="token function" style="color:#d73a49">loadPluginStore</span><span class="token punctuation" style="color:#393A34">(</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">        </span><span class="token class-name">IPluginContext</span><span class="token plain"> pluginContext</span><span class="token punctuation" style="color:#393A34">,</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">        </span><span class="token class-name">ManipulateItemsProcessor</span><span class="token plain"> itemsProcessor</span><span class="token punctuation" style="color:#393A34">)</span><span class="token plain"> </span><span class="token punctuation" style="color:#393A34">{</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">    </span><span class="token comment" style="color:#999988;font-style:italic">// 从 pluginMeta 中取出业务上下文</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">    </span><span class="token class-name">String</span><span class="token plain"> domain </span><span class="token operator" style="color:#393A34">=</span><span class="token plain"> itemsProcessor</span><span class="token punctuation" style="color:#393A34">.</span><span class="token function" style="color:#d73a49">getPluginMeta</span><span class="token punctuation" style="color:#393A34">(</span><span class="token punctuation" style="color:#393A34">)</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">            </span><span class="token punctuation" style="color:#393A34">.</span><span class="token function" style="color:#d73a49">getExtraParam</span><span class="token punctuation" style="color:#393A34">(</span><span class="token class-name">OntologyDomain</span><span class="token punctuation" style="color:#393A34">.</span><span class="token constant" style="color:#36acaa">NAME_ONTOLOGY_DOMAIN</span><span class="token punctuation" style="color:#393A34">)</span><span class="token punctuation" style="color:#393A34">;</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">    </span><span class="token class-name">KeyedPluginStore</span><span class="token class-name punctuation" style="color:#393A34">.</span><span class="token class-name">Key</span><span class="token generics punctuation" style="color:#393A34">&lt;</span><span class="token generics class-name">MyManipulate</span><span class="token generics punctuation" style="color:#393A34">&gt;</span><span class="token plain"> key </span><span class="token operator" style="color:#393A34">=</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">            </span><span class="token class-name">OntologyDomain</span><span class="token punctuation" style="color:#393A34">.</span><span class="token function" style="color:#d73a49">getStoreKey</span><span class="token punctuation" style="color:#393A34">(</span><span class="token plain">domain</span><span class="token punctuation" style="color:#393A34">,</span><span class="token plain"> </span><span class="token class-name">MyManipulate</span><span class="token punctuation" style="color:#393A34">.</span><span class="token keyword" style="color:#00009f">class</span><span class="token punctuation" style="color:#393A34">)</span><span class="token punctuation" style="color:#393A34">;</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">    </span><span class="token keyword" style="color:#00009f">return</span><span class="token plain"> </span><span class="token constant" style="color:#36acaa">TIS</span><span class="token punctuation" style="color:#393A34">.</span><span class="token function" style="color:#d73a49">getPluginStore</span><span class="token punctuation" style="color:#393A34">(</span><span class="token plain">key</span><span class="token punctuation" style="color:#393A34">)</span><span class="token punctuation" style="color:#393A34">;</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain"></span><span class="token punctuation" style="color:#393A34">}</span><br></span></code></pre><div class="buttonGroup__atx"><button type="button" aria-label="Copy code to clipboard" title="Copy" class="clean-btn"><span class="copyButtonIcons_eSgA" aria-hidden="true"><svg class="copyButtonIcon_y97N" viewBox="0 0 24 24"><path d="M19,21H8V7H19M19,5H8A2,2 0 0,0 6,7V21A2,2 0 0,0 8,23H19A2,2 0 0,0 21,21V7A2,2 0 0,0 19,5M16,1H4A2,2 0 0,0 2,3V17H4V3H16V1Z"></path></svg><svg class="copyButtonSuccessIcon_LjdS" viewBox="0 0 24 24"><path d="M21,7L9,19L3.5,13.5L4.91,12.09L9,16.17L19.59,5.59L21,7Z"></path></svg></span></button></div></div></div><h4 class="anchor anchorWithStickyNavbar_LWe7" id="aftermanipuldateprocess按需覆盖"><code>afterManipuldateProcess()</code>（按需覆盖）<a class="hash-link" href="#aftermanipuldateprocess按需覆盖" title="Direct link to heading">​</a></h4><p>持久化完成后的后续操作。默认是空实现，如果你的操作需要在保存后触发额外逻辑（如同步到 Neo4j、发送通知），在这里写。</p><div class="language-java codeBlockContainer_Ckt0 theme-code-block" style="--prism-color:#393A34;--prism-background-color:#f6f8fa"><div class="codeBlockContent_biex"><pre tabindex="0" class="prism-code language-java codeBlock_bY9V thin-scrollbar"><code class="codeBlockLines_e6Vv"><span class="token-line" style="color:#393A34"><span class="token annotation punctuation" style="color:#393A34">@Override</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain"></span><span class="token keyword" style="color:#00009f">protected</span><span class="token plain"> </span><span class="token keyword" style="color:#00009f">void</span><span class="token plain"> </span><span class="token function" style="color:#d73a49">afterManipuldateProcess</span><span class="token punctuation" style="color:#393A34">(</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">        </span><span class="token class-name">IPluginContext</span><span class="token plain"> pluginContext</span><span class="token punctuation" style="color:#393A34">,</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">        </span><span class="token class-name">Optional</span><span class="token generics punctuation" style="color:#393A34">&lt;</span><span class="token generics class-name">Context</span><span class="token generics punctuation" style="color:#393A34">&gt;</span><span class="token plain"> context</span><span class="token punctuation" style="color:#393A34">,</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">        </span><span class="token class-name">ManipulateItemsProcessor</span><span class="token plain"> itemsProcessor</span><span class="token punctuation" style="color:#393A34">)</span><span class="token plain"> </span><span class="token punctuation" style="color:#393A34">{</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">    </span><span class="token keyword" style="color:#00009f">if</span><span class="token plain"> </span><span class="token punctuation" style="color:#393A34">(</span><span class="token plain">itemsProcessor</span><span class="token punctuation" style="color:#393A34">.</span><span class="token function" style="color:#d73a49">isDeleteProcess</span><span class="token punctuation" style="color:#393A34">(</span><span class="token punctuation" style="color:#393A34">)</span><span class="token punctuation" style="color:#393A34">)</span><span class="token plain"> </span><span class="token punctuation" style="color:#393A34">{</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">        </span><span class="token keyword" style="color:#00009f">return</span><span class="token punctuation" style="color:#393A34">;</span><span class="token plain"> </span><span class="token comment" style="color:#999988;font-style:italic">// 删除时通常不需要后续操作</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">    </span><span class="token punctuation" style="color:#393A34">}</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">    </span><span class="token comment" style="color:#999988;font-style:italic">// 触发你的业务逻辑</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">    </span><span class="token class-name">MyService</span><span class="token punctuation" style="color:#393A34">.</span><span class="token function" style="color:#d73a49">getInstance</span><span class="token punctuation" style="color:#393A34">(</span><span class="token punctuation" style="color:#393A34">)</span><span class="token punctuation" style="color:#393A34">.</span><span class="token function" style="color:#d73a49">doSomething</span><span class="token punctuation" style="color:#393A34">(</span><span class="token punctuation" style="color:#393A34">)</span><span class="token punctuation" style="color:#393A34">;</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain"></span><span class="token punctuation" style="color:#393A34">}</span><br></span></code></pre><div class="buttonGroup__atx"><button type="button" aria-label="Copy code to clipboard" title="Copy" class="clean-btn"><span class="copyButtonIcons_eSgA" aria-hidden="true"><svg class="copyButtonIcon_y97N" viewBox="0 0 24 24"><path d="M19,21H8V7H19M19,5H8A2,2 0 0,0 6,7V21A2,2 0 0,0 8,23H19A2,2 0 0,0 21,21V7A2,2 0 0,0 19,5M16,1H4A2,2 0 0,0 2,3V17H4V3H16V1Z"></path></svg><svg class="copyButtonSuccessIcon_LjdS" viewBox="0 0 24 24"><path d="M21,7L9,19L3.5,13.5L4.91,12.09L9,16.17L19.59,5.59L21,7Z"></path></svg></span></button></div></div></div><hr><h3 class="anchor anchorWithStickyNavbar_LWe7" id="33-basicdesc内部-descriptor-基类">3.3 <code>BasicDesc</code>（内部 Descriptor 基类）<a class="hash-link" href="#33-basicdesc内部-descriptor-基类" title="Direct link to heading">​</a></h3><p>每个 Manipulate 插件都需要一个内部静态类 <code>DftDesc</code>（或其他名字），继承对应的 <code>BasicDesc</code>，并加上 <code>@TISExtension</code> 注解。</p><p><strong>最关键的方法：<code>isManipulateStorable()</code></strong></p><div class="language-java codeBlockContainer_Ckt0 theme-code-block" style="--prism-color:#393A34;--prism-background-color:#f6f8fa"><div class="codeBlockContent_biex"><pre tabindex="0" class="prism-code language-java codeBlock_bY9V thin-scrollbar"><code class="codeBlockLines_e6Vv"><span class="token-line" style="color:#393A34"><span class="token annotation punctuation" style="color:#393A34">@TISExtension</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain"></span><span class="token keyword" style="color:#00009f">public</span><span class="token plain"> </span><span class="token keyword" style="color:#00009f">static</span><span class="token plain"> </span><span class="token keyword" style="color:#00009f">final</span><span class="token plain"> </span><span class="token keyword" style="color:#00009f">class</span><span class="token plain"> </span><span class="token class-name">DftDesc</span><span class="token plain"> </span><span class="token keyword" style="color:#00009f">extends</span><span class="token plain"> </span><span class="token class-name">OntologyDomainManipulate</span><span class="token class-name punctuation" style="color:#393A34">.</span><span class="token class-name">BasicDesc</span><span class="token plain"> </span><span class="token punctuation" style="color:#393A34">{</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">    </span><span class="token annotation punctuation" style="color:#393A34">@Override</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">    </span><span class="token keyword" style="color:#00009f">public</span><span class="token plain"> </span><span class="token keyword" style="color:#00009f">boolean</span><span class="token plain"> </span><span class="token function" style="color:#d73a49">isManipulateStorable</span><span class="token punctuation" style="color:#393A34">(</span><span class="token punctuation" style="color:#393A34">)</span><span class="token plain"> </span><span class="token punctuation" style="color:#393A34">{</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">        </span><span class="token keyword" style="color:#00009f">return</span><span class="token plain"> </span><span class="token boolean" style="color:#36acaa">true</span><span class="token punctuation" style="color:#393A34">;</span><span class="token plain">  </span><span class="token comment" style="color:#999988;font-style:italic">// true = 需要持久化；false = 只执行逻辑，不存储</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">    </span><span class="token punctuation" style="color:#393A34">}</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain"></span><span class="token punctuation" style="color:#393A34">}</span><br></span></code></pre><div class="buttonGroup__atx"><button type="button" aria-label="Copy code to clipboard" title="Copy" class="clean-btn"><span class="copyButtonIcons_eSgA" aria-hidden="true"><svg class="copyButtonIcon_y97N" viewBox="0 0 24 24"><path d="M19,21H8V7H19M19,5H8A2,2 0 0,0 6,7V21A2,2 0 0,0 8,23H19A2,2 0 0,0 21,21V7A2,2 0 0,0 19,5M16,1H4A2,2 0 0,0 2,3V17H4V3H16V1Z"></path></svg><svg class="copyButtonSuccessIcon_LjdS" viewBox="0 0 24 24"><path d="M21,7L9,19L3.5,13.5L4.91,12.09L9,16.17L19.59,5.59L21,7Z"></path></svg></span></button></div></div></div><table><thead><tr><th><code>isManipulateStorable()</code> 返回值</th><th>行为</th></tr></thead><tbody><tr><td><code>true</code></td><td>基类自动完成判重、持久化（<code>replaceInStore</code>/<code>deleteFromStore</code>），然后调用 <code>afterManipuldateProcess</code></td></tr><tr><td><code>false</code></td><td>跳过持久化，直接调用 <code>afterManipuldateProcess</code></td></tr></tbody></table><hr><h2 class="anchor anchorWithStickyNavbar_LWe7" id="四开发流程手把手写一个-manipulate-插件">四、开发流程：手把手写一个 Manipulate 插件<a class="hash-link" href="#四开发流程手把手写一个-manipulate-插件" title="Direct link to heading">​</a></h2><p>以"为本体域开启某个新功能"为例，完整步骤如下。</p><h3 class="anchor anchorWithStickyNavbar_LWe7" id="step-1确定你的操作属于哪个核心实体">Step 1：确定你的操作属于哪个核心实体<a class="hash-link" href="#step-1确定你的操作属于哪个核心实体" title="Direct link to heading">​</a></h3><ul><li>操作针对 <strong>DataX 数据管道</strong> → 继承 <code>DefaultDataXProcessorManipulate</code></li><li>操作针对 <strong>本体域</strong> → 继承 <code>OntologyDomainManipulate</code></li><li>操作针对其他实体 → 继承对应的 <code>BasicManipuldateProcessor</code> 子类</li></ul><h3 class="anchor anchorWithStickyNavbar_LWe7" id="step-2创建操作类">Step 2：创建操作类<a class="hash-link" href="#step-2创建操作类" title="Direct link to heading">​</a></h3><div class="language-java codeBlockContainer_Ckt0 theme-code-block" style="--prism-color:#393A34;--prism-background-color:#f6f8fa"><div class="codeBlockContent_biex"><pre tabindex="0" class="prism-code language-java codeBlock_bY9V thin-scrollbar"><code class="codeBlockLines_e6Vv"><span class="token-line" style="color:#393A34"><span class="token keyword" style="color:#00009f">package</span><span class="token plain"> </span><span class="token namespace" style="opacity:0.7">com</span><span class="token namespace punctuation" style="opacity:0.7;color:#393A34">.</span><span class="token namespace" style="opacity:0.7">qlangtech</span><span class="token namespace punctuation" style="opacity:0.7;color:#393A34">.</span><span class="token namespace" style="opacity:0.7">tis</span><span class="token namespace punctuation" style="opacity:0.7;color:#393A34">.</span><span class="token namespace" style="opacity:0.7">plugin</span><span class="token namespace punctuation" style="opacity:0.7;color:#393A34">.</span><span class="token namespace" style="opacity:0.7">ontology</span><span class="token punctuation" style="color:#393A34">;</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain" style="display:inline-block"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain"></span><span class="token keyword" style="color:#00009f">import</span><span class="token plain"> </span><span class="token import namespace" style="opacity:0.7">com</span><span class="token import namespace punctuation" style="opacity:0.7;color:#393A34">.</span><span class="token import namespace" style="opacity:0.7">qlangtech</span><span class="token import namespace punctuation" style="opacity:0.7;color:#393A34">.</span><span class="token import namespace" style="opacity:0.7">tis</span><span class="token import namespace punctuation" style="opacity:0.7;color:#393A34">.</span><span class="token import namespace" style="opacity:0.7">extension</span><span class="token import namespace punctuation" style="opacity:0.7;color:#393A34">.</span><span class="token import class-name">TISExtension</span><span class="token punctuation" style="color:#393A34">;</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain"></span><span class="token keyword" style="color:#00009f">import</span><span class="token plain"> </span><span class="token import namespace" style="opacity:0.7">com</span><span class="token import namespace punctuation" style="opacity:0.7;color:#393A34">.</span><span class="token import namespace" style="opacity:0.7">qlangtech</span><span class="token import namespace punctuation" style="opacity:0.7;color:#393A34">.</span><span class="token import namespace" style="opacity:0.7">tis</span><span class="token import namespace punctuation" style="opacity:0.7;color:#393A34">.</span><span class="token import namespace" style="opacity:0.7">plugin</span><span class="token import namespace punctuation" style="opacity:0.7;color:#393A34">.</span><span class="token import namespace" style="opacity:0.7">annotation</span><span class="token import namespace punctuation" style="opacity:0.7;color:#393A34">.</span><span class="token import class-name">FormField</span><span class="token punctuation" style="color:#393A34">;</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain"></span><span class="token keyword" style="color:#00009f">import</span><span class="token plain"> </span><span class="token import namespace" style="opacity:0.7">com</span><span class="token import namespace punctuation" style="opacity:0.7;color:#393A34">.</span><span class="token import namespace" style="opacity:0.7">qlangtech</span><span class="token import namespace punctuation" style="opacity:0.7;color:#393A34">.</span><span class="token import namespace" style="opacity:0.7">tis</span><span class="token import namespace punctuation" style="opacity:0.7;color:#393A34">.</span><span class="token import namespace" style="opacity:0.7">plugin</span><span class="token import namespace punctuation" style="opacity:0.7;color:#393A34">.</span><span class="token import namespace" style="opacity:0.7">annotation</span><span class="token import namespace punctuation" style="opacity:0.7;color:#393A34">.</span><span class="token import class-name">FormFieldType</span><span class="token punctuation" style="color:#393A34">;</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain"></span><span class="token keyword" style="color:#00009f">import</span><span class="token plain"> </span><span class="token import namespace" style="opacity:0.7">com</span><span class="token import namespace punctuation" style="opacity:0.7;color:#393A34">.</span><span class="token import namespace" style="opacity:0.7">qlangtech</span><span class="token import namespace punctuation" style="opacity:0.7;color:#393A34">.</span><span class="token import namespace" style="opacity:0.7">tis</span><span class="token import namespace punctuation" style="opacity:0.7;color:#393A34">.</span><span class="token import namespace" style="opacity:0.7">plugin</span><span class="token import namespace punctuation" style="opacity:0.7;color:#393A34">.</span><span class="token import namespace" style="opacity:0.7">annotation</span><span class="token import namespace punctuation" style="opacity:0.7;color:#393A34">.</span><span class="token import class-name">Validator</span><span class="token punctuation" style="color:#393A34">;</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain"></span><span class="token keyword" style="color:#00009f">import</span><span class="token plain"> </span><span class="token import namespace" style="opacity:0.7">com</span><span class="token import namespace punctuation" style="opacity:0.7;color:#393A34">.</span><span class="token import namespace" style="opacity:0.7">qlangtech</span><span class="token import namespace punctuation" style="opacity:0.7;color:#393A34">.</span><span class="token import namespace" style="opacity:0.7">tis</span><span class="token import namespace punctuation" style="opacity:0.7;color:#393A34">.</span><span class="token import namespace" style="opacity:0.7">plugin</span><span class="token import namespace punctuation" style="opacity:0.7;color:#393A34">.</span><span class="token import namespace" style="opacity:0.7">ds</span><span class="token import namespace punctuation" style="opacity:0.7;color:#393A34">.</span><span class="token import namespace" style="opacity:0.7">manipulate</span><span class="token import namespace punctuation" style="opacity:0.7;color:#393A34">.</span><span class="token import class-name">ManipulateItemsProcessor</span><span class="token punctuation" style="color:#393A34">;</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain"></span><span class="token keyword" style="color:#00009f">import</span><span class="token plain"> </span><span class="token import namespace" style="opacity:0.7">com</span><span class="token import namespace punctuation" style="opacity:0.7;color:#393A34">.</span><span class="token import namespace" style="opacity:0.7">qlangtech</span><span class="token import namespace punctuation" style="opacity:0.7;color:#393A34">.</span><span class="token import namespace" style="opacity:0.7">tis</span><span class="token import namespace punctuation" style="opacity:0.7;color:#393A34">.</span><span class="token import namespace" style="opacity:0.7">runtime</span><span class="token import namespace punctuation" style="opacity:0.7;color:#393A34">.</span><span class="token import namespace" style="opacity:0.7">module</span><span class="token import namespace punctuation" style="opacity:0.7;color:#393A34">.</span><span class="token import namespace" style="opacity:0.7">misc</span><span class="token import namespace punctuation" style="opacity:0.7;color:#393A34">.</span><span class="token import class-name">IControlMsgHandler</span><span class="token punctuation" style="color:#393A34">;</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain"></span><span class="token keyword" style="color:#00009f">import</span><span class="token plain"> </span><span class="token import namespace" style="opacity:0.7">com</span><span class="token import namespace punctuation" style="opacity:0.7;color:#393A34">.</span><span class="token import namespace" style="opacity:0.7">qlangtech</span><span class="token import namespace punctuation" style="opacity:0.7;color:#393A34">.</span><span class="token import namespace" style="opacity:0.7">tis</span><span class="token import namespace punctuation" style="opacity:0.7;color:#393A34">.</span><span class="token import namespace" style="opacity:0.7">util</span><span class="token import namespace punctuation" style="opacity:0.7;color:#393A34">.</span><span class="token import class-name">IPluginContext</span><span class="token punctuation" style="color:#393A34">;</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain" style="display:inline-block"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain"></span><span class="token keyword" style="color:#00009f">import</span><span class="token plain"> </span><span class="token import namespace" style="opacity:0.7">java</span><span class="token import namespace punctuation" style="opacity:0.7;color:#393A34">.</span><span class="token import namespace" style="opacity:0.7">util</span><span class="token import namespace punctuation" style="opacity:0.7;color:#393A34">.</span><span class="token import class-name">Optional</span><span class="token punctuation" style="color:#393A34">;</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain"></span><span class="token keyword" style="color:#00009f">import</span><span class="token plain"> </span><span class="token import namespace" style="opacity:0.7">com</span><span class="token import namespace punctuation" style="opacity:0.7;color:#393A34">.</span><span class="token import namespace" style="opacity:0.7">alibaba</span><span class="token import namespace punctuation" style="opacity:0.7;color:#393A34">.</span><span class="token import namespace" style="opacity:0.7">citrus</span><span class="token import namespace punctuation" style="opacity:0.7;color:#393A34">.</span><span class="token import namespace" style="opacity:0.7">turbine</span><span class="token import namespace punctuation" style="opacity:0.7;color:#393A34">.</span><span class="token import class-name">Context</span><span class="token punctuation" style="color:#393A34">;</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain" style="display:inline-block"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain"></span><span class="token keyword" style="color:#00009f">public</span><span class="token plain"> </span><span class="token keyword" style="color:#00009f">class</span><span class="token plain"> </span><span class="token class-name">MyNewFeature</span><span class="token plain"> </span><span class="token keyword" style="color:#00009f">extends</span><span class="token plain"> </span><span class="token class-name">OntologyDomainManipulate</span><span class="token plain"> </span><span class="token punctuation" style="color:#393A34">{</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain" style="display:inline-block"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">    </span><span class="token comment" style="color:#999988;font-style:italic">// 在界面上展示的表单字段</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">    </span><span class="token annotation punctuation" style="color:#393A34">@FormField</span><span class="token punctuation" style="color:#393A34">(</span><span class="token plain">ordinal </span><span class="token operator" style="color:#393A34">=</span><span class="token plain"> </span><span class="token number" style="color:#36acaa">1</span><span class="token punctuation" style="color:#393A34">,</span><span class="token plain"> type </span><span class="token operator" style="color:#393A34">=</span><span class="token plain"> </span><span class="token class-name">FormFieldType</span><span class="token punctuation" style="color:#393A34">.</span><span class="token constant" style="color:#36acaa">INPUTTEXT</span><span class="token punctuation" style="color:#393A34">,</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">               validate </span><span class="token operator" style="color:#393A34">=</span><span class="token plain"> </span><span class="token punctuation" style="color:#393A34">{</span><span class="token class-name">Validator</span><span class="token punctuation" style="color:#393A34">.</span><span class="token plain">require</span><span class="token punctuation" style="color:#393A34">}</span><span class="token punctuation" style="color:#393A34">)</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">    </span><span class="token keyword" style="color:#00009f">public</span><span class="token plain"> </span><span class="token class-name">String</span><span class="token plain"> someConfig</span><span class="token punctuation" style="color:#393A34">;</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain" style="display:inline-block"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">    </span><span class="token annotation punctuation" style="color:#393A34">@Override</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">    </span><span class="token keyword" style="color:#00009f">protected</span><span class="token plain"> </span><span class="token keyword" style="color:#00009f">void</span><span class="token plain"> </span><span class="token function" style="color:#d73a49">afterManipuldateProcess</span><span class="token punctuation" style="color:#393A34">(</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">            </span><span class="token class-name">IPluginContext</span><span class="token plain"> pluginContext</span><span class="token punctuation" style="color:#393A34">,</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">            </span><span class="token class-name">Optional</span><span class="token generics punctuation" style="color:#393A34">&lt;</span><span class="token generics class-name">Context</span><span class="token generics punctuation" style="color:#393A34">&gt;</span><span class="token plain"> context</span><span class="token punctuation" style="color:#393A34">,</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">            </span><span class="token class-name">ManipulateItemsProcessor</span><span class="token plain"> itemsProcessor</span><span class="token punctuation" style="color:#393A34">)</span><span class="token plain"> </span><span class="token punctuation" style="color:#393A34">{</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">        </span><span class="token keyword" style="color:#00009f">if</span><span class="token plain"> </span><span class="token punctuation" style="color:#393A34">(</span><span class="token plain">itemsProcessor</span><span class="token punctuation" style="color:#393A34">.</span><span class="token function" style="color:#d73a49">isDeleteProcess</span><span class="token punctuation" style="color:#393A34">(</span><span class="token punctuation" style="color:#393A34">)</span><span class="token punctuation" style="color:#393A34">)</span><span class="token plain"> </span><span class="token punctuation" style="color:#393A34">{</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">            </span><span class="token keyword" style="color:#00009f">return</span><span class="token punctuation" style="color:#393A34">;</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">        </span><span class="token punctuation" style="color:#393A34">}</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">        </span><span class="token comment" style="color:#999988;font-style:italic">// 持久化完成后，触发你的业务逻辑</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">        </span><span class="token class-name">String</span><span class="token plain"> domain </span><span class="token operator" style="color:#393A34">=</span><span class="token plain"> itemsProcessor</span><span class="token punctuation" style="color:#393A34">.</span><span class="token function" style="color:#d73a49">getPluginMeta</span><span class="token punctuation" style="color:#393A34">(</span><span class="token punctuation" style="color:#393A34">)</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">                </span><span class="token punctuation" style="color:#393A34">.</span><span class="token function" style="color:#d73a49">getExtraParam</span><span class="token punctuation" style="color:#393A34">(</span><span class="token class-name">OntologyDomain</span><span class="token punctuation" style="color:#393A34">.</span><span class="token constant" style="color:#36acaa">NAME_ONTOLOGY_DOMAIN</span><span class="token punctuation" style="color:#393A34">)</span><span class="token punctuation" style="color:#393A34">;</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">        </span><span class="token class-name">MyService</span><span class="token punctuation" style="color:#393A34">.</span><span class="token function" style="color:#d73a49">activate</span><span class="token punctuation" style="color:#393A34">(</span><span class="token plain">domain</span><span class="token punctuation" style="color:#393A34">,</span><span class="token plain"> </span><span class="token keyword" style="color:#00009f">this</span><span class="token punctuation" style="color:#393A34">.</span><span class="token plain">someConfig</span><span class="token punctuation" style="color:#393A34">)</span><span class="token punctuation" style="color:#393A34">;</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">    </span><span class="token punctuation" style="color:#393A34">}</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain" style="display:inline-block"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">    </span><span class="token annotation punctuation" style="color:#393A34">@TISExtension</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">    </span><span class="token keyword" style="color:#00009f">public</span><span class="token plain"> </span><span class="token keyword" style="color:#00009f">static</span><span class="token plain"> </span><span class="token keyword" style="color:#00009f">final</span><span class="token plain"> </span><span class="token keyword" style="color:#00009f">class</span><span class="token plain"> </span><span class="token class-name">DftDesc</span><span class="token plain"> </span><span class="token keyword" style="color:#00009f">extends</span><span class="token plain"> </span><span class="token class-name">OntologyDomainManipulate</span><span class="token class-name punctuation" style="color:#393A34">.</span><span class="token class-name">BasicDesc</span><span class="token plain"> </span><span class="token punctuation" style="color:#393A34">{</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">        </span><span class="token keyword" style="color:#00009f">public</span><span class="token plain"> </span><span class="token class-name">DftDesc</span><span class="token punctuation" style="color:#393A34">(</span><span class="token punctuation" style="color:#393A34">)</span><span class="token plain"> </span><span class="token punctuation" style="color:#393A34">{</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">            </span><span class="token keyword" style="color:#00009f">super</span><span class="token punctuation" style="color:#393A34">(</span><span class="token punctuation" style="color:#393A34">)</span><span class="token punctuation" style="color:#393A34">;</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">        </span><span class="token punctuation" style="color:#393A34">}</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain" style="display:inline-block"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">        </span><span class="token annotation punctuation" style="color:#393A34">@Override</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">        </span><span class="token keyword" style="color:#00009f">public</span><span class="token plain"> </span><span class="token keyword" style="color:#00009f">boolean</span><span class="token plain"> </span><span class="token function" style="color:#d73a49">isManipulateStorable</span><span class="token punctuation" style="color:#393A34">(</span><span class="token punctuation" style="color:#393A34">)</span><span class="token plain"> </span><span class="token punctuation" style="color:#393A34">{</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">            </span><span class="token keyword" style="color:#00009f">return</span><span class="token plain"> </span><span class="token boolean" style="color:#36acaa">true</span><span class="token punctuation" style="color:#393A34">;</span><span class="token plain"> </span><span class="token comment" style="color:#999988;font-style:italic">// 需要持久化</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">        </span><span class="token punctuation" style="color:#393A34">}</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain" style="display:inline-block"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">        </span><span class="token annotation punctuation" style="color:#393A34">@Override</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">        </span><span class="token keyword" style="color:#00009f">public</span><span class="token plain"> </span><span class="token class-name">String</span><span class="token plain"> </span><span class="token function" style="color:#d73a49">getDisplayName</span><span class="token punctuation" style="color:#393A34">(</span><span class="token punctuation" style="color:#393A34">)</span><span class="token plain"> </span><span class="token punctuation" style="color:#393A34">{</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">            </span><span class="token keyword" style="color:#00009f">return</span><span class="token plain"> </span><span class="token string" style="color:#e3116c">"My New Feature"</span><span class="token punctuation" style="color:#393A34">;</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">        </span><span class="token punctuation" style="color:#393A34">}</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">    </span><span class="token punctuation" style="color:#393A34">}</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain"></span><span class="token punctuation" style="color:#393A34">}</span><br></span></code></pre><div class="buttonGroup__atx"><button type="button" aria-label="Copy code to clipboard" title="Copy" class="clean-btn"><span class="copyButtonIcons_eSgA" aria-hidden="true"><svg class="copyButtonIcon_y97N" viewBox="0 0 24 24"><path d="M19,21H8V7H19M19,5H8A2,2 0 0,0 6,7V21A2,2 0 0,0 8,23H19A2,2 0 0,0 21,21V7A2,2 0 0,0 19,5M16,1H4A2,2 0 0,0 2,3V17H4V3H16V1Z"></path></svg><svg class="copyButtonSuccessIcon_LjdS" viewBox="0 0 24 24"><path d="M21,7L9,19L3.5,13.5L4.91,12.09L9,16.17L19.59,5.59L21,7Z"></path></svg></span></button></div></div></div><h3 class="anchor anchorWithStickyNavbar_LWe7" id="step-3判断是否需要持久化">Step 3：判断是否需要持久化<a class="hash-link" href="#step-3判断是否需要持久化" title="Direct link to heading">​</a></h3><p>问自己一个问题：<strong>用户下次打开界面，需要看到这个操作的配置吗？</strong></p><ul><li><strong>需要</strong>（如 <code>EnableChatBI</code> 的 LLM 选择）→ <code>isManipulateStorable()</code> 返回 <code>true</code></li><li><strong>不需要</strong>（如 <code>InferOntologyFromLLM</code> 只是触发一次推断）→ <code>isManipulateStorable()</code> 返回 <code>false</code></li></ul><h3 class="anchor anchorWithStickyNavbar_LWe7" id="step-4处理删除逻辑">Step 4：处理删除逻辑<a class="hash-link" href="#step-4处理删除逻辑" title="Direct link to heading">​</a></h3><p><code>afterManipuldateProcess</code> 在删除操作时也会被调用。通过 <code>itemsProcessor.isDeleteProcess()</code> 判断：</p><div class="language-java codeBlockContainer_Ckt0 theme-code-block" style="--prism-color:#393A34;--prism-background-color:#f6f8fa"><div class="codeBlockContent_biex"><pre tabindex="0" class="prism-code language-java codeBlock_bY9V thin-scrollbar"><code class="codeBlockLines_e6Vv"><span class="token-line" style="color:#393A34"><span class="token annotation punctuation" style="color:#393A34">@Override</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain"></span><span class="token keyword" style="color:#00009f">protected</span><span class="token plain"> </span><span class="token keyword" style="color:#00009f">void</span><span class="token plain"> </span><span class="token function" style="color:#d73a49">afterManipuldateProcess</span><span class="token punctuation" style="color:#393A34">(</span><span class="token punctuation" style="color:#393A34">.</span><span class="token punctuation" style="color:#393A34">.</span><span class="token punctuation" style="color:#393A34">.</span><span class="token punctuation" style="color:#393A34">)</span><span class="token plain"> </span><span class="token punctuation" style="color:#393A34">{</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">    </span><span class="token keyword" style="color:#00009f">if</span><span class="token plain"> </span><span class="token punctuation" style="color:#393A34">(</span><span class="token plain">itemsProcessor</span><span class="token punctuation" style="color:#393A34">.</span><span class="token function" style="color:#d73a49">isDeleteProcess</span><span class="token punctuation" style="color:#393A34">(</span><span class="token punctuation" style="color:#393A34">)</span><span class="token punctuation" style="color:#393A34">)</span><span class="token plain"> </span><span class="token punctuation" style="color:#393A34">{</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">        </span><span class="token comment" style="color:#999988;font-style:italic">// 清理资源，如关闭连接、删除远端配置</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">        </span><span class="token class-name">MyService</span><span class="token punctuation" style="color:#393A34">.</span><span class="token function" style="color:#d73a49">deactivate</span><span class="token punctuation" style="color:#393A34">(</span><span class="token plain">domain</span><span class="token punctuation" style="color:#393A34">)</span><span class="token punctuation" style="color:#393A34">;</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">        </span><span class="token keyword" style="color:#00009f">return</span><span class="token punctuation" style="color:#393A34">;</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">    </span><span class="token punctuation" style="color:#393A34">}</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">    </span><span class="token comment" style="color:#999988;font-style:italic">// 新增/更新逻辑</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">    </span><span class="token class-name">MyService</span><span class="token punctuation" style="color:#393A34">.</span><span class="token function" style="color:#d73a49">activate</span><span class="token punctuation" style="color:#393A34">(</span><span class="token plain">domain</span><span class="token punctuation" style="color:#393A34">,</span><span class="token plain"> </span><span class="token keyword" style="color:#00009f">this</span><span class="token punctuation" style="color:#393A34">.</span><span class="token plain">someConfig</span><span class="token punctuation" style="color:#393A34">)</span><span class="token punctuation" style="color:#393A34">;</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain"></span><span class="token punctuation" style="color:#393A34">}</span><br></span></code></pre><div class="buttonGroup__atx"><button type="button" aria-label="Copy code to clipboard" title="Copy" class="clean-btn"><span class="copyButtonIcons_eSgA" aria-hidden="true"><svg class="copyButtonIcon_y97N" viewBox="0 0 24 24"><path d="M19,21H8V7H19M19,5H8A2,2 0 0,0 6,7V21A2,2 0 0,0 8,23H19A2,2 0 0,0 21,21V7A2,2 0 0,0 19,5M16,1H4A2,2 0 0,0 2,3V17H4V3H16V1Z"></path></svg><svg class="copyButtonSuccessIcon_LjdS" viewBox="0 0 24 24"><path d="M21,7L9,19L3.5,13.5L4.91,12.09L9,16.17L19.59,5.59L21,7Z"></path></svg></span></button></div></div></div><hr><h2 class="anchor anchorWithStickyNavbar_LWe7" id="五两个典型案例对比">五、两个典型案例对比<a class="hash-link" href="#五两个典型案例对比" title="Direct link to heading">​</a></h2><h3 class="anchor anchorWithStickyNavbar_LWe7" id="案例-aenablechatbi需持久化--有后续操作">案例 A：<code>EnableChatBI</code>（需持久化 + 有后续操作）<a class="hash-link" href="#案例-aenablechatbi需持久化--有后续操作" title="Direct link to heading">​</a></h3><div class="language-java codeBlockContainer_Ckt0 theme-code-block" style="--prism-color:#393A34;--prism-background-color:#f6f8fa"><div class="codeBlockContent_biex"><pre tabindex="0" class="prism-code language-java codeBlock_bY9V thin-scrollbar"><code class="codeBlockLines_e6Vv"><span class="token-line" style="color:#393A34"><span class="token keyword" style="color:#00009f">public</span><span class="token plain"> </span><span class="token keyword" style="color:#00009f">class</span><span class="token plain"> </span><span class="token class-name">EnableChatBI</span><span class="token plain"> </span><span class="token keyword" style="color:#00009f">extends</span><span class="token plain"> </span><span class="token class-name">OntologyDomainManipulate</span><span class="token plain"> </span><span class="token keyword" style="color:#00009f">implements</span><span class="token plain"> </span><span class="token class-name">IManipulateStatus</span><span class="token plain"> </span><span class="token punctuation" style="color:#393A34">{</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain" style="display:inline-block"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">    </span><span class="token annotation punctuation" style="color:#393A34">@FormField</span><span class="token punctuation" style="color:#393A34">(</span><span class="token plain">type </span><span class="token operator" style="color:#393A34">=</span><span class="token plain"> </span><span class="token class-name">FormFieldType</span><span class="token punctuation" style="color:#393A34">.</span><span class="token constant" style="color:#36acaa">SELECTABLE</span><span class="token punctuation" style="color:#393A34">,</span><span class="token plain"> ordinal </span><span class="token operator" style="color:#393A34">=</span><span class="token plain"> </span><span class="token number" style="color:#36acaa">1</span><span class="token punctuation" style="color:#393A34">,</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">               validate </span><span class="token operator" style="color:#393A34">=</span><span class="token plain"> </span><span class="token punctuation" style="color:#393A34">{</span><span class="token class-name">Validator</span><span class="token punctuation" style="color:#393A34">.</span><span class="token plain">identity</span><span class="token punctuation" style="color:#393A34">,</span><span class="token plain"> </span><span class="token class-name">Validator</span><span class="token punctuation" style="color:#393A34">.</span><span class="token plain">require</span><span class="token punctuation" style="color:#393A34">}</span><span class="token punctuation" style="color:#393A34">)</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">    </span><span class="token keyword" style="color:#00009f">public</span><span class="token plain"> </span><span class="token class-name">String</span><span class="token plain"> llm</span><span class="token punctuation" style="color:#393A34">;</span><span class="token plain">  </span><span class="token comment" style="color:#999988;font-style:italic">// 用户选择的大模型</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain" style="display:inline-block"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">    </span><span class="token annotation punctuation" style="color:#393A34">@Override</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">    </span><span class="token keyword" style="color:#00009f">protected</span><span class="token plain"> </span><span class="token keyword" style="color:#00009f">void</span><span class="token plain"> </span><span class="token function" style="color:#d73a49">afterManipuldateProcess</span><span class="token punctuation" style="color:#393A34">(</span><span class="token class-name">IPluginContext</span><span class="token plain"> pluginContext</span><span class="token punctuation" style="color:#393A34">,</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">            </span><span class="token class-name">Optional</span><span class="token generics punctuation" style="color:#393A34">&lt;</span><span class="token generics class-name">Context</span><span class="token generics punctuation" style="color:#393A34">&gt;</span><span class="token plain"> context</span><span class="token punctuation" style="color:#393A34">,</span><span class="token plain"> </span><span class="token class-name">ManipulateItemsProcessor</span><span class="token plain"> itemsProcessor</span><span class="token punctuation" style="color:#393A34">)</span><span class="token plain"> </span><span class="token punctuation" style="color:#393A34">{</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">        </span><span class="token keyword" style="color:#00009f">if</span><span class="token plain"> </span><span class="token punctuation" style="color:#393A34">(</span><span class="token plain">itemsProcessor</span><span class="token punctuation" style="color:#393A34">.</span><span class="token function" style="color:#d73a49">isDeleteProcess</span><span class="token punctuation" style="color:#393A34">(</span><span class="token punctuation" style="color:#393A34">)</span><span class="token punctuation" style="color:#393A34">)</span><span class="token plain"> </span><span class="token keyword" style="color:#00009f">return</span><span class="token punctuation" style="color:#393A34">;</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">        </span><span class="token comment" style="color:#999988;font-style:italic">// 持久化完成后，触发 Neo4j 全量重建</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">        </span><span class="token class-name">String</span><span class="token plain"> domainName </span><span class="token operator" style="color:#393A34">=</span><span class="token plain"> itemsProcessor</span><span class="token punctuation" style="color:#393A34">.</span><span class="token function" style="color:#d73a49">getPluginMeta</span><span class="token punctuation" style="color:#393A34">(</span><span class="token punctuation" style="color:#393A34">)</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">                </span><span class="token punctuation" style="color:#393A34">.</span><span class="token function" style="color:#d73a49">getExtraParam</span><span class="token punctuation" style="color:#393A34">(</span><span class="token class-name">OntologyDomain</span><span class="token punctuation" style="color:#393A34">.</span><span class="token constant" style="color:#36acaa">NAME_ONTOLOGY_DOMAIN</span><span class="token punctuation" style="color:#393A34">)</span><span class="token punctuation" style="color:#393A34">;</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">        </span><span class="token class-name">OntologySyncQueue</span><span class="token punctuation" style="color:#393A34">.</span><span class="token function" style="color:#d73a49">enqueue</span><span class="token punctuation" style="color:#393A34">(</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">            </span><span class="token punctuation" style="color:#393A34">(</span><span class="token punctuation" style="color:#393A34">)</span><span class="token plain"> </span><span class="token operator" style="color:#393A34">-&gt;</span><span class="token plain"> </span><span class="token class-name">OntologyNeo4jSyncService</span><span class="token punctuation" style="color:#393A34">.</span><span class="token function" style="color:#d73a49">getInstance</span><span class="token punctuation" style="color:#393A34">(</span><span class="token punctuation" style="color:#393A34">)</span><span class="token punctuation" style="color:#393A34">.</span><span class="token function" style="color:#d73a49">fullRebuild</span><span class="token punctuation" style="color:#393A34">(</span><span class="token plain">domainName</span><span class="token punctuation" style="color:#393A34">)</span><span class="token punctuation" style="color:#393A34">)</span><span class="token punctuation" style="color:#393A34">;</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">    </span><span class="token punctuation" style="color:#393A34">}</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain" style="display:inline-block"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">    </span><span class="token annotation punctuation" style="color:#393A34">@TISExtension</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">    </span><span class="token keyword" style="color:#00009f">public</span><span class="token plain"> </span><span class="token keyword" style="color:#00009f">static</span><span class="token plain"> </span><span class="token keyword" style="color:#00009f">final</span><span class="token plain"> </span><span class="token keyword" style="color:#00009f">class</span><span class="token plain"> </span><span class="token class-name">DftDesc</span><span class="token plain"> </span><span class="token keyword" style="color:#00009f">extends</span><span class="token plain"> </span><span class="token class-name">OntologyDomainManipulate</span><span class="token class-name punctuation" style="color:#393A34">.</span><span class="token class-name">BasicDesc</span><span class="token plain"> </span><span class="token punctuation" style="color:#393A34">{</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">        </span><span class="token annotation punctuation" style="color:#393A34">@Override</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">        </span><span class="token keyword" style="color:#00009f">public</span><span class="token plain"> </span><span class="token keyword" style="color:#00009f">boolean</span><span class="token plain"> </span><span class="token function" style="color:#d73a49">isManipulateStorable</span><span class="token punctuation" style="color:#393A34">(</span><span class="token punctuation" style="color:#393A34">)</span><span class="token plain"> </span><span class="token punctuation" style="color:#393A34">{</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">            </span><span class="token keyword" style="color:#00009f">return</span><span class="token plain"> </span><span class="token boolean" style="color:#36acaa">true</span><span class="token punctuation" style="color:#393A34">;</span><span class="token plain"> </span><span class="token comment" style="color:#999988;font-style:italic">// llm 字段需要持久化，下次打开界面能看到</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">        </span><span class="token punctuation" style="color:#393A34">}</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">    </span><span class="token punctuation" style="color:#393A34">}</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain"></span><span class="token punctuation" style="color:#393A34">}</span><br></span></code></pre><div class="buttonGroup__atx"><button type="button" aria-label="Copy code to clipboard" title="Copy" class="clean-btn"><span class="copyButtonIcons_eSgA" aria-hidden="true"><svg class="copyButtonIcon_y97N" viewBox="0 0 24 24"><path d="M19,21H8V7H19M19,5H8A2,2 0 0,0 6,7V21A2,2 0 0,0 8,23H19A2,2 0 0,0 21,21V7A2,2 0 0,0 19,5M16,1H4A2,2 0 0,0 2,3V17H4V3H16V1Z"></path></svg><svg class="copyButtonSuccessIcon_LjdS" viewBox="0 0 24 24"><path d="M21,7L9,19L3.5,13.5L4.91,12.09L9,16.17L19.59,5.59L21,7Z"></path></svg></span></button></div></div></div><p><strong>要点：</strong></p><ul><li><code>isManipulateStorable() = true</code>：<code>llm</code> 字段会被保存到 XML 配置文件</li><li><code>afterManipuldateProcess</code> 里触发异步 Neo4j 同步</li><li>实现了 <code>IManipulateStatus</code> 接口，可以在界面上展示当前状态</li></ul><hr><h3 class="anchor anchorWithStickyNavbar_LWe7" id="案例-binferontologyfromllm不持久化只执行一次性任务">案例 B：<code>InferOntologyFromLLM</code>（不持久化，只执行一次性任务）<a class="hash-link" href="#案例-binferontologyfromllm不持久化只执行一次性任务" title="Direct link to heading">​</a></h3><div class="language-java codeBlockContainer_Ckt0 theme-code-block" style="--prism-color:#393A34;--prism-background-color:#f6f8fa"><div class="codeBlockContent_biex"><pre tabindex="0" class="prism-code language-java codeBlock_bY9V thin-scrollbar"><code class="codeBlockLines_e6Vv"><span class="token-line" style="color:#393A34"><span class="token keyword" style="color:#00009f">public</span><span class="token plain"> </span><span class="token keyword" style="color:#00009f">class</span><span class="token plain"> </span><span class="token class-name">InferOntologyFromLLM</span><span class="token plain"> </span><span class="token keyword" style="color:#00009f">extends</span><span class="token plain"> </span><span class="token class-name">OntologyDomainManipulate</span><span class="token plain"> </span><span class="token punctuation" style="color:#393A34">{</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain" style="display:inline-block"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">    </span><span class="token annotation punctuation" style="color:#393A34">@FormField</span><span class="token punctuation" style="color:#393A34">(</span><span class="token plain">type </span><span class="token operator" style="color:#393A34">=</span><span class="token plain"> </span><span class="token class-name">FormFieldType</span><span class="token punctuation" style="color:#393A34">.</span><span class="token constant" style="color:#36acaa">SELECTABLE</span><span class="token punctuation" style="color:#393A34">,</span><span class="token plain"> ordinal </span><span class="token operator" style="color:#393A34">=</span><span class="token plain"> </span><span class="token number" style="color:#36acaa">1</span><span class="token punctuation" style="color:#393A34">)</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">    </span><span class="token keyword" style="color:#00009f">public</span><span class="token plain"> </span><span class="token class-name">String</span><span class="token plain"> llm</span><span class="token punctuation" style="color:#393A34">;</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain" style="display:inline-block"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">    </span><span class="token annotation punctuation" style="color:#393A34">@Override</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">    </span><span class="token keyword" style="color:#00009f">protected</span><span class="token plain"> </span><span class="token keyword" style="color:#00009f">void</span><span class="token plain"> </span><span class="token function" style="color:#d73a49">afterManipuldateProcess</span><span class="token punctuation" style="color:#393A34">(</span><span class="token class-name">IPluginContext</span><span class="token plain"> pluginContext</span><span class="token punctuation" style="color:#393A34">,</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">            </span><span class="token class-name">Optional</span><span class="token generics punctuation" style="color:#393A34">&lt;</span><span class="token generics class-name">Context</span><span class="token generics punctuation" style="color:#393A34">&gt;</span><span class="token plain"> context</span><span class="token punctuation" style="color:#393A34">,</span><span class="token plain"> </span><span class="token class-name">ManipulateItemsProcessor</span><span class="token plain"> itemsProcessor</span><span class="token punctuation" style="color:#393A34">)</span><span class="token plain"> </span><span class="token punctuation" style="color:#393A34">{</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">        </span><span class="token comment" style="color:#999988;font-style:italic">// 调用 LLM 推断本体关系，结果直接写入本体存储</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">        </span><span class="token class-name">OntologyPluginMeta</span><span class="token plain"> ometa </span><span class="token operator" style="color:#393A34">=</span><span class="token plain"> </span><span class="token function" style="color:#d73a49">getOntologyPluginMeta</span><span class="token punctuation" style="color:#393A34">(</span><span class="token plain">pluginContext</span><span class="token punctuation" style="color:#393A34">,</span><span class="token plain"> context</span><span class="token punctuation" style="color:#393A34">)</span><span class="token punctuation" style="color:#393A34">;</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">        </span><span class="token class-name">List</span><span class="token generics punctuation" style="color:#393A34">&lt;</span><span class="token generics class-name">OntologyObjectType</span><span class="token generics punctuation" style="color:#393A34">&gt;</span><span class="token plain"> objectTypes </span><span class="token operator" style="color:#393A34">=</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">                </span><span class="token class-name">OntologyObjectType</span><span class="token punctuation" style="color:#393A34">.</span><span class="token function" style="color:#d73a49">loadAll</span><span class="token punctuation" style="color:#393A34">(</span><span class="token plain">ometa</span><span class="token punctuation" style="color:#393A34">.</span><span class="token function" style="color:#d73a49">getDomain</span><span class="token punctuation" style="color:#393A34">(</span><span class="token punctuation" style="color:#393A34">)</span><span class="token punctuation" style="color:#393A34">)</span><span class="token punctuation" style="color:#393A34">;</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">        </span><span class="token comment" style="color:#999988;font-style:italic">// ... 调用 LLM，解析结果，写入本体 ...</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">    </span><span class="token punctuation" style="color:#393A34">}</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain" style="display:inline-block"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">    </span><span class="token annotation punctuation" style="color:#393A34">@TISExtension</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">    </span><span class="token keyword" style="color:#00009f">public</span><span class="token plain"> </span><span class="token keyword" style="color:#00009f">static</span><span class="token plain"> </span><span class="token keyword" style="color:#00009f">final</span><span class="token plain"> </span><span class="token keyword" style="color:#00009f">class</span><span class="token plain"> </span><span class="token class-name">DftDesc</span><span class="token plain"> </span><span class="token keyword" style="color:#00009f">extends</span><span class="token plain"> </span><span class="token class-name">OntologyDomainManipulate</span><span class="token class-name punctuation" style="color:#393A34">.</span><span class="token class-name">BasicDesc</span><span class="token plain"> </span><span class="token punctuation" style="color:#393A34">{</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">        </span><span class="token annotation punctuation" style="color:#393A34">@Override</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">        </span><span class="token keyword" style="color:#00009f">public</span><span class="token plain"> </span><span class="token keyword" style="color:#00009f">boolean</span><span class="token plain"> </span><span class="token function" style="color:#d73a49">isManipulateStorable</span><span class="token punctuation" style="color:#393A34">(</span><span class="token punctuation" style="color:#393A34">)</span><span class="token plain"> </span><span class="token punctuation" style="color:#393A34">{</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">            </span><span class="token keyword" style="color:#00009f">return</span><span class="token plain"> </span><span class="token boolean" style="color:#36acaa">false</span><span class="token punctuation" style="color:#393A34">;</span><span class="token plain"> </span><span class="token comment" style="color:#999988;font-style:italic">// 不需要持久化，每次触发都是全新推断</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">        </span><span class="token punctuation" style="color:#393A34">}</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">    </span><span class="token punctuation" style="color:#393A34">}</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain"></span><span class="token punctuation" style="color:#393A34">}</span><br></span></code></pre><div class="buttonGroup__atx"><button type="button" aria-label="Copy code to clipboard" title="Copy" class="clean-btn"><span class="copyButtonIcons_eSgA" aria-hidden="true"><svg class="copyButtonIcon_y97N" viewBox="0 0 24 24"><path d="M19,21H8V7H19M19,5H8A2,2 0 0,0 6,7V21A2,2 0 0,0 8,23H19A2,2 0 0,0 21,21V7A2,2 0 0,0 19,5M16,1H4A2,2 0 0,0 2,3V17H4V3H16V1Z"></path></svg><svg class="copyButtonSuccessIcon_LjdS" viewBox="0 0 24 24"><path d="M21,7L9,19L3.5,13.5L4.91,12.09L9,16.17L19.59,5.59L21,7Z"></path></svg></span></button></div></div></div><p><strong>要点：</strong></p><ul><li><code>isManipulateStorable() = false</code>：基类跳过持久化，直接调用 <code>afterManipuldateProcess</code></li><li>所有业务逻辑都在 <code>afterManipuldateProcess</code> 里</li></ul><hr><h2 class="anchor anchorWithStickyNavbar_LWe7" id="六常见错误与注意事项">六、常见错误与注意事项<a class="hash-link" href="#六常见错误与注意事项" title="Direct link to heading">​</a></h2><h3 class="anchor anchorWithStickyNavbar_LWe7" id="-错误-1覆盖-manipuldateprocess">❌ 错误 1：覆盖 <code>manipuldateProcess</code><a class="hash-link" href="#-错误-1覆盖-manipuldateprocess" title="Direct link to heading">​</a></h3><div class="language-java codeBlockContainer_Ckt0 theme-code-block" style="--prism-color:#393A34;--prism-background-color:#f6f8fa"><div class="codeBlockContent_biex"><pre tabindex="0" class="prism-code language-java codeBlock_bY9V thin-scrollbar"><code class="codeBlockLines_e6Vv"><span class="token-line" style="color:#393A34"><span class="token comment" style="color:#999988;font-style:italic">// 错误！manipuldateProcess 是 final 的，不能覆盖</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain"></span><span class="token annotation punctuation" style="color:#393A34">@Override</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain"></span><span class="token keyword" style="color:#00009f">public</span><span class="token plain"> </span><span class="token keyword" style="color:#00009f">void</span><span class="token plain"> </span><span class="token function" style="color:#d73a49">manipuldateProcess</span><span class="token punctuation" style="color:#393A34">(</span><span class="token punctuation" style="color:#393A34">.</span><span class="token punctuation" style="color:#393A34">.</span><span class="token punctuation" style="color:#393A34">.</span><span class="token punctuation" style="color:#393A34">)</span><span class="token plain"> </span><span class="token punctuation" style="color:#393A34">{</span><span class="token plain"> </span><span class="token punctuation" style="color:#393A34">.</span><span class="token punctuation" style="color:#393A34">.</span><span class="token punctuation" style="color:#393A34">.</span><span class="token plain"> </span><span class="token punctuation" style="color:#393A34">}</span><br></span></code></pre><div class="buttonGroup__atx"><button type="button" aria-label="Copy code to clipboard" title="Copy" class="clean-btn"><span class="copyButtonIcons_eSgA" aria-hidden="true"><svg class="copyButtonIcon_y97N" viewBox="0 0 24 24"><path d="M19,21H8V7H19M19,5H8A2,2 0 0,0 6,7V21A2,2 0 0,0 8,23H19A2,2 0 0,0 21,21V7A2,2 0 0,0 19,5M16,1H4A2,2 0 0,0 2,3V17H4V3H16V1Z"></path></svg><svg class="copyButtonSuccessIcon_LjdS" viewBox="0 0 24 24"><path d="M21,7L9,19L3.5,13.5L4.91,12.09L9,16.17L19.59,5.59L21,7Z"></path></svg></span></button></div></div></div><p><strong>正确做法</strong>：把逻辑放到 <code>afterManipuldateProcess</code>。</p><hr><h3 class="anchor anchorWithStickyNavbar_LWe7" id="-错误-2dftdesc-忘记加-tisextension">❌ 错误 2：<code>DftDesc</code> 忘记加 <code>@TISExtension</code><a class="hash-link" href="#-错误-2dftdesc-忘记加-tisextension" title="Direct link to heading">​</a></h3><div class="language-java codeBlockContainer_Ckt0 theme-code-block" style="--prism-color:#393A34;--prism-background-color:#f6f8fa"><div class="codeBlockContent_biex"><pre tabindex="0" class="prism-code language-java codeBlock_bY9V thin-scrollbar"><code class="codeBlockLines_e6Vv"><span class="token-line" style="color:#393A34"><span class="token comment" style="color:#999988;font-style:italic">// 错误！没有 @TISExtension，TIS 框架找不到这个 Descriptor</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain"></span><span class="token keyword" style="color:#00009f">public</span><span class="token plain"> </span><span class="token keyword" style="color:#00009f">static</span><span class="token plain"> </span><span class="token keyword" style="color:#00009f">final</span><span class="token plain"> </span><span class="token keyword" style="color:#00009f">class</span><span class="token plain"> </span><span class="token class-name">DftDesc</span><span class="token plain"> </span><span class="token keyword" style="color:#00009f">extends</span><span class="token plain"> </span><span class="token class-name">OntologyDomainManipulate</span><span class="token class-name punctuation" style="color:#393A34">.</span><span class="token class-name">BasicDesc</span><span class="token plain"> </span><span class="token punctuation" style="color:#393A34">{</span><span class="token plain"> </span><span class="token punctuation" style="color:#393A34">.</span><span class="token punctuation" style="color:#393A34">.</span><span class="token punctuation" style="color:#393A34">.</span><span class="token plain"> </span><span class="token punctuation" style="color:#393A34">}</span><br></span></code></pre><div class="buttonGroup__atx"><button type="button" aria-label="Copy code to clipboard" title="Copy" class="clean-btn"><span class="copyButtonIcons_eSgA" aria-hidden="true"><svg class="copyButtonIcon_y97N" viewBox="0 0 24 24"><path d="M19,21H8V7H19M19,5H8A2,2 0 0,0 6,7V21A2,2 0 0,0 8,23H19A2,2 0 0,0 21,21V7A2,2 0 0,0 19,5M16,1H4A2,2 0 0,0 2,3V17H4V3H16V1Z"></path></svg><svg class="copyButtonSuccessIcon_LjdS" viewBox="0 0 24 24"><path d="M21,7L9,19L3.5,13.5L4.91,12.09L9,16.17L19.59,5.59L21,7Z"></path></svg></span></button></div></div></div><p><strong>正确做法</strong>：必须加 <code>@TISExtension</code>。</p><hr><h3 class="anchor anchorWithStickyNavbar_LWe7" id="-错误-3dftdesc-没有继承正确的-basicdesc">❌ 错误 3：<code>DftDesc</code> 没有继承正确的 <code>BasicDesc</code><a class="hash-link" href="#-错误-3dftdesc-没有继承正确的-basicdesc" title="Direct link to heading">​</a></h3><div class="language-java codeBlockContainer_Ckt0 theme-code-block" style="--prism-color:#393A34;--prism-background-color:#f6f8fa"><div class="codeBlockContent_biex"><pre tabindex="0" class="prism-code language-java codeBlock_bY9V thin-scrollbar"><code class="codeBlockLines_e6Vv"><span class="token-line" style="color:#393A34"><span class="token comment" style="color:#999988;font-style:italic">// 错误！继承了错误的基类</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain"></span><span class="token keyword" style="color:#00009f">public</span><span class="token plain"> </span><span class="token keyword" style="color:#00009f">static</span><span class="token plain"> </span><span class="token keyword" style="color:#00009f">final</span><span class="token plain"> </span><span class="token keyword" style="color:#00009f">class</span><span class="token plain"> </span><span class="token class-name">DftDesc</span><span class="token plain"> </span><span class="token keyword" style="color:#00009f">extends</span><span class="token plain"> </span><span class="token class-name">Descriptor</span><span class="token generics punctuation" style="color:#393A34">&lt;</span><span class="token generics class-name">MyFeature</span><span class="token generics punctuation" style="color:#393A34">&gt;</span><span class="token plain"> </span><span class="token punctuation" style="color:#393A34">{</span><span class="token plain"> </span><span class="token punctuation" style="color:#393A34">.</span><span class="token punctuation" style="color:#393A34">.</span><span class="token punctuation" style="color:#393A34">.</span><span class="token plain"> </span><span class="token punctuation" style="color:#393A34">}</span><br></span></code></pre><div class="buttonGroup__atx"><button type="button" aria-label="Copy code to clipboard" title="Copy" class="clean-btn"><span class="copyButtonIcons_eSgA" aria-hidden="true"><svg class="copyButtonIcon_y97N" viewBox="0 0 24 24"><path d="M19,21H8V7H19M19,5H8A2,2 0 0,0 6,7V21A2,2 0 0,0 8,23H19A2,2 0 0,0 21,21V7A2,2 0 0,0 19,5M16,1H4A2,2 0 0,0 2,3V17H4V3H16V1Z"></path></svg><svg class="copyButtonSuccessIcon_LjdS" viewBox="0 0 24 24"><path d="M21,7L9,19L3.5,13.5L4.91,12.09L9,16.17L19.59,5.59L21,7Z"></path></svg></span></button></div></div></div><p><strong>正确做法</strong>：必须继承对应的 <code>BasicDesc</code>（<code>OntologyDomainManipulate.BasicDesc</code> 或 <code>DefaultDataXProcessorManipulate.BasicDesc</code>）。</p><hr><h3 class="anchor anchorWithStickyNavbar_LWe7" id="-错误-4在-aftermanipuldateprocess-里忘记处理删除分支">❌ 错误 4：在 <code>afterManipuldateProcess</code> 里忘记处理删除分支<a class="hash-link" href="#-错误-4在-aftermanipuldateprocess-里忘记处理删除分支" title="Direct link to heading">​</a></h3><div class="language-java codeBlockContainer_Ckt0 theme-code-block" style="--prism-color:#393A34;--prism-background-color:#f6f8fa"><div class="codeBlockContent_biex"><pre tabindex="0" class="prism-code language-java codeBlock_bY9V thin-scrollbar"><code class="codeBlockLines_e6Vv"><span class="token-line" style="color:#393A34"><span class="token comment" style="color:#999988;font-style:italic">// 危险！删除时也会触发 afterManipuldateProcess</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain"></span><span class="token annotation punctuation" style="color:#393A34">@Override</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain"></span><span class="token keyword" style="color:#00009f">protected</span><span class="token plain"> </span><span class="token keyword" style="color:#00009f">void</span><span class="token plain"> </span><span class="token function" style="color:#d73a49">afterManipuldateProcess</span><span class="token punctuation" style="color:#393A34">(</span><span class="token punctuation" style="color:#393A34">.</span><span class="token punctuation" style="color:#393A34">.</span><span class="token punctuation" style="color:#393A34">.</span><span class="token punctuation" style="color:#393A34">)</span><span class="token plain"> </span><span class="token punctuation" style="color:#393A34">{</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">    </span><span class="token comment" style="color:#999988;font-style:italic">// 如果是删除操作，这里会报错（domain 已不存在）</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">    </span><span class="token class-name">MyService</span><span class="token punctuation" style="color:#393A34">.</span><span class="token function" style="color:#d73a49">activate</span><span class="token punctuation" style="color:#393A34">(</span><span class="token plain">domain</span><span class="token punctuation" style="color:#393A34">)</span><span class="token punctuation" style="color:#393A34">;</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain"></span><span class="token punctuation" style="color:#393A34">}</span><br></span></code></pre><div class="buttonGroup__atx"><button type="button" aria-label="Copy code to clipboard" title="Copy" class="clean-btn"><span class="copyButtonIcons_eSgA" aria-hidden="true"><svg class="copyButtonIcon_y97N" viewBox="0 0 24 24"><path d="M19,21H8V7H19M19,5H8A2,2 0 0,0 6,7V21A2,2 0 0,0 8,23H19A2,2 0 0,0 21,21V7A2,2 0 0,0 19,5M16,1H4A2,2 0 0,0 2,3V17H4V3H16V1Z"></path></svg><svg class="copyButtonSuccessIcon_LjdS" viewBox="0 0 24 24"><path d="M21,7L9,19L3.5,13.5L4.91,12.09L9,16.17L19.59,5.59L21,7Z"></path></svg></span></button></div></div></div><p><strong>正确做法</strong>：先判断 <code>itemsProcessor.isDeleteProcess()</code>。</p><hr><h3 class="anchor anchorWithStickyNavbar_LWe7" id="️-注意loadpluginstore-中的上下文提取">⚠️ 注意：<code>loadPluginStore</code> 中的上下文提取<a class="hash-link" href="#️-注意loadpluginstore-中的上下文提取" title="Direct link to heading">​</a></h3><p><code>itemsProcessor.getPluginMeta().getExtraParam(key)</code> 是从请求的 <code>pluginMeta</code> 中提取参数。不同的核心实体，参数 key 不同：</p><table><thead><tr><th>核心实体</th><th>上下文 key</th><th>获取方式</th></tr></thead><tbody><tr><td>本体域</td><td><code>OntologyDomain.NAME_ONTOLOGY_DOMAIN</code></td><td><code>pluginMeta.getExtraParam(...)</code></td></tr><tr><td>DataX 管道</td><td><code>DataXName</code></td><td><code>itemsProcessor.getOriginIdentityId()</code></td></tr></tbody></table><hr><h2 class="anchor anchorWithStickyNavbar_LWe7" id="七开发检查清单">七、开发检查清单<a class="hash-link" href="#七开发检查清单" title="Direct link to heading">​</a></h2><p>开发完成后，对照以下清单自查：</p><ul class="contains-task-list containsTaskList_mC6p"><li class="task-list-item"><input type="checkbox" disabled=""> <!-- -->操作类继承了正确的基类（<code>OntologyDomainManipulate</code> 或 <code>DefaultDataXProcessorManipulate</code>）</li><li class="task-list-item"><input type="checkbox" disabled=""> <!-- -->实现了 <code>loadPluginStore()</code>（如果直接继承 <code>BasicManipuldateProcessor</code>）</li><li class="task-list-item"><input type="checkbox" disabled=""> <!-- -->内部类 <code>DftDesc</code> 加了 <code>@TISExtension</code></li><li class="task-list-item"><input type="checkbox" disabled=""> <code>DftDesc</code> 继承了正确的 <code>BasicDesc</code></li><li class="task-list-item"><input type="checkbox" disabled=""> <code>isManipulateStorable()</code> 返回值与业务需求一致</li><li class="task-list-item"><input type="checkbox" disabled=""> <code>afterManipuldateProcess</code> 里处理了删除分支</li><li class="task-list-item"><input type="checkbox" disabled=""> <!-- -->编译通过：<code>mvn compile -q</code></li></ul><hr><h2 class="anchor anchorWithStickyNavbar_LWe7" id="八总结">八、总结<a class="hash-link" href="#八总结" title="Direct link to heading">​</a></h2><table><thead><tr><th>场景</th><th>做法</th></tr></thead><tbody><tr><td>需要持久化配置</td><td><code>isManipulateStorable()</code> 返回 <code>true</code>，业务逻辑写在 <code>afterManipuldateProcess</code></td></tr><tr><td>只执行一次性任务</td><td><code>isManipulateStorable()</code> 返回 <code>false</code>，逻辑全写在 <code>afterManipuldateProcess</code></td></tr><tr><td>持久化后需要触发额外操作</td><td>覆盖 <code>afterManipuldateProcess</code>，在里面调用业务服务</td></tr><tr><td>克隆/重命名场景</td><td>覆盖 <code>getNewIdentityName()</code> 返回新名称</td></tr></tbody></table><p>整个体系的设计思路是<strong>模板方法模式</strong>：基类 <code>BasicManipuldateProcessor</code> 定义了持久化的完整流程，子类只需填充"存到哪里"（<code>loadPluginStore</code>）和"存完之后做什么"（<code>afterManipuldateProcess</code>）两个空白，其余的校验、判重、存储、删除逻辑由框架统一处理。</p><p>有问题随时找团队的同学，欢迎加入 TIS！</p>]]></content>
        <author>
            <name>百岁</name>
            <uri>https://github.com/baisui1981</uri>
        </author>
        <category label="tis" term="tis"/>
        <category label="plugin" term="plugin"/>
        <category label="manipulate" term="manipulate"/>
        <category label="开发指南" term="开发指南"/>
    </entry>
    <entry>
        <title type="html"><![CDATA[如何优雅地解决复杂配置问题——TIS多步配置插件架构实践]]></title>
        <id>/2026/01/28/multi-step-plugin</id>
        <link href="https://tis.pub/blog/2026/01/28/multi-step-plugin"/>
        <updated>2026-01-28T00:00:00.000Z</updated>
        <summary type="html"><![CDATA[引言]]></summary>
        <content type="html"><![CDATA[<h2 class="anchor anchorWithStickyNavbar_LWe7" id="引言">引言<a class="hash-link" href="#引言" title="Direct link to heading">​</a></h2><p>在数据集成平台的开发过程中，我们经常会遇到这样的场景：某个功能的配置项非常多，涉及多个维度的选择，而且这些配置项之间存在依赖关系。比如配置主表与维表的JOIN关系时，需要先选择数据源，再选择目标表，最后设置匹配条件和输出列。如果把所有配置项都堆在一个页面上，不仅用户体验差，而且代码维护困难。</p><p>传统的单页面配置方式存在明显的局限性：配置项过多导致页面臃肿、用户容易迷失在复杂的表单中、配置步骤的逻辑关系不清晰。更重要的是，当需要新增类似的复杂配置功能时，往往需要重复大量的开发工作。</p><p>TIS（Table Integration System）通过设计一套通用的多步配置插件架构，优雅地解决了这个问题。这套架构不仅让复杂配置变得简单易用，更重要的是，它具有极强的扩展性——新增一个多步配置功能，只需要实现几个接口，编写少量代码即可完成。本文将深入剖析这套架构的设计原理，帮助开发者理解如何构建一个既灵活又易于扩展的配置系统。</p><h2 class="anchor anchorWithStickyNavbar_LWe7" id="多步配置插件的核心架构">多步配置插件的核心架构<a class="hash-link" href="#多步配置插件的核心架构" title="Direct link to heading">​</a></h2><h3 class="anchor anchorWithStickyNavbar_LWe7" id="整体架构设计">整体架构设计<a class="hash-link" href="#整体架构设计" title="Direct link to heading">​</a></h3><p>TIS的多步配置插件架构采用了前后端分离的设计思想，通过清晰的接口定义和配置驱动的方式，实现了高度的解耦和可扩展性。</p><p><strong>【图片位置1：多步配置插件整体架构图】</strong>
<em>说明：展示前端（Angular组件）、后端（Java插件接口）、配置文件（JSON）三者之间的关系，以及数据流转的方向</em></p><p>整个架构可以分为三个核心层次：</p><ol><li><strong>插件宿主层</strong>：定义多步配置的容器，负责管理所有步骤</li><li><strong>步骤实现层</strong>：每个步骤的具体配置逻辑</li><li><strong>数据解析层</strong>：将前端提交的数据转换为后端对象</li></ol><p>这三层通过接口进行交互，每一层都可以独立扩展，互不影响。</p><h3 class="anchor anchorWithStickyNavbar_LWe7" id="后端架构设计">后端架构设计<a class="hash-link" href="#后端架构设计" title="Direct link to heading">​</a></h3><p>后端架构的核心是一套精心设计的接口体系，它定义了多步配置的标准流程和扩展点。</p><h4 class="anchor anchorWithStickyNavbar_LWe7" id="multistepssupporthost多步配置的宿主">MultiStepsSupportHost：多步配置的宿主<a class="hash-link" href="#multistepssupporthost多步配置的宿主" title="Direct link to heading">​</a></h4><p>这是多步配置插件的顶层接口，任何需要支持多步配置的插件都需要实现这个接口。它的职责非常明确：</p><ul><li>持有所有步骤的配置数据</li><li>提供步骤数据的访问接口</li><li>定义步骤的元数据（步骤数量、每步的描述等）</li></ul><p>通过实现这个接口，一个普通的插件就具备了多步配置的能力。这种设计遵循了"组合优于继承"的原则，让多步配置成为一种可选的能力，而不是强制的约束。</p><h4 class="anchor anchorWithStickyNavbar_LWe7" id="onestepofmultisteps单步配置的抽象">OneStepOfMultiSteps：单步配置的抽象<a class="hash-link" href="#onestepofmultisteps单步配置的抽象" title="Direct link to heading">​</a></h4><p>每个配置步骤都是一个独立的插件，继承自OneStepOfMultiSteps抽象类。这个设计非常巧妙：</p><ul><li>每个步骤都是独立的，有自己的配置项和验证逻辑</li><li>步骤之间通过接口方法进行数据传递</li><li>步骤可以访问前序步骤的配置数据，实现配置的级联</li></ul><p>例如，在JOIN配置场景中，第一步选择数据源后，第二步就可以根据选择的数据源加载对应的表列表；第三步又可以根据前两步的选择，展示可用的列信息。这种设计让复杂的配置流程变得清晰可控。</p><h4 class="anchor anchorWithStickyNavbar_LWe7" id="elementcreatorfactory数据解析的工厂模式">ElementCreatorFactory：数据解析的工厂模式<a class="hash-link" href="#elementcreatorfactory数据解析的工厂模式" title="Direct link to heading">​</a></h4><p>当用户在前端填写了复杂的表单数据（比如多个匹配条件、多个过滤规则），这些数据需要被解析成后端的Java对象。ElementCreatorFactory就是专门负责这个转换过程的工厂类。</p><p>它的设计体现了几个关键思想：</p><ol><li><strong>职责单一</strong>：每种数据类型对应一个Factory，只负责解析自己的数据</li><li><strong>配置驱动</strong>：通过JSON配置文件指定使用哪个Factory，无需修改代码</li><li><strong>验证集成</strong>：在解析过程中同步进行数据验证，及时反馈错误</li></ol><p>更重要的是，Factory还负责向前端提供元数据，比如下拉框的选项、字段的约束条件等。这种双向的数据交互机制，让前后端的协作变得非常流畅。</p><h4 class="anchor anchorWithStickyNavbar_LWe7" id="viewcontent前后端视图类型的桥梁">ViewContent：前后端视图类型的桥梁<a class="hash-link" href="#viewcontent前后端视图类型的桥梁" title="Direct link to heading">​</a></h4><p>ViewContent是一个枚举类型，它定义了所有支持的视图类型。每种视图类型对应一种特定的数据结构和UI组件。</p><p>这个设计看似简单，实则非常关键：</p><ul><li>它是前后端约定的"协议"，双方通过ViewContent来识别数据类型</li><li>新增一种视图类型，只需要在枚举中添加一个值</li><li>前后端可以独立开发，只要遵循ViewContent的约定即可</li></ul><p><strong>【图片位置2：前后端数据流转示意图】</strong>
<em>说明：展示用户操作 → 前端组件 → ViewContent映射 → ElementCreatorFactory解析 → 后端对象的完整流程</em></p><h3 class="anchor anchorWithStickyNavbar_LWe7" id="前端架构设计">前端架构设计<a class="hash-link" href="#前端架构设计" title="Direct link to heading">​</a></h3><p>前端架构同样遵循了组件化和类型驱动的设计原则。</p><h4 class="anchor anchorWithStickyNavbar_LWe7" id="tuplespropertytype视图类型的注册机制">TuplesPropertyType：视图类型的注册机制<a class="hash-link" href="#tuplespropertytype视图类型的注册机制" title="Direct link to heading">​</a></h4><p>TuplesPropertyType是前端对应ViewContent的枚举类型。当后端返回配置数据时，会携带ViewContent信息，前端通过TuplesPropertyType进行映射，决定使用哪个组件来渲染。</p><p>这种类型注册机制的优势在于：</p><ul><li>新增视图类型时，只需要在switch语句中添加一个case分支</li><li>每个组件都是独立的，可以单独开发和测试</li><li>类型安全，编译期就能发现错误</li></ul><h4 class="anchor anchorWithStickyNavbar_LWe7" id="组件化的步骤ui">组件化的步骤UI<a class="hash-link" href="#组件化的步骤ui" title="Direct link to heading">​</a></h4><p>每个配置步骤对应一个Angular组件，这些组件都继承自BasicTuplesViewComponent基类。基类提供了通用的功能：</p><ul><li>数据绑定和验证</li><li>错误信息展示</li><li>与父组件的通信</li></ul><p>具体的步骤组件只需要关注自己的业务逻辑，比如渲染表单、处理用户交互等。这种设计让组件的复用性非常高。</p><h4 class="anchor anchorWithStickyNavbar_LWe7" id="数据双向绑定与验证">数据双向绑定与验证<a class="hash-link" href="#数据双向绑定与验证" title="Direct link to heading">​</a></h4><p>前端使用Angular的双向绑定机制，用户的每次输入都会实时同步到数据模型中。当用户提交配置时，前端会将数据序列化为JSON格式，发送给后端。</p><p>验证逻辑分为两层：</p><ol><li><strong>前端验证</strong>：即时反馈，提升用户体验</li><li><strong>后端验证</strong>：保证数据安全，防止恶意提交</li></ol><p>后端验证失败时，错误信息会通过统一的格式返回给前端，前端根据字段路径将错误信息显示在对应的表单项旁边。</p><h2 class="anchor anchorWithStickyNavbar_LWe7" id="架构的扩展性设计">架构的扩展性设计<a class="hash-link" href="#架构的扩展性设计" title="Direct link to heading">​</a></h2><p>这套架构最大的价值在于它的扩展性。当需要新增一个多步配置功能时，开发工作量非常小，这得益于几个关键的设计决策。</p><h3 class="anchor anchorWithStickyNavbar_LWe7" id="接口抽象的威力">接口抽象的威力<a class="hash-link" href="#接口抽象的威力" title="Direct link to heading">​</a></h3><p>通过定义清晰的接口，架构将"做什么"和"怎么做"完全分离。</p><p>以JOIN配置为例，当我们需要新增"过滤条件"功能时，只需要：</p><ol><li>创建一个新的数据类TableJoinFilterCondition，实现IMultiElement接口</li><li>创建对应的Factory类，实现ElementCreatorFactory接口</li><li>在ViewContent枚举中添加一个新值</li></ol><p>这三步完成后，后端的扩展就完成了。整个过程不需要修改任何现有代码，完全符合"开闭原则"——对扩展开放，对修改关闭。</p><p>接口抽象的另一个好处是，它强制开发者思考功能的本质。比如ElementCreatorFactory接口定义了几个关键方法：</p><ul><li>parsePostMCols：解析前端提交的数据</li><li>appendExternalJsonProp：向前端提供元数据</li><li>getViewContentType：返回视图类型</li></ul><p>任何需要解析复杂表单数据的场景，都可以套用这个模式。开发者不需要重新设计数据流转的逻辑，只需要实现这几个方法即可。</p><h3 class="anchor anchorWithStickyNavbar_LWe7" id="配置驱动的灵活性">配置驱动的灵活性<a class="hash-link" href="#配置驱动的灵活性" title="Direct link to heading">​</a></h3><p>JSON配置文件在这套架构中扮演了"粘合剂"的角色。它连接了Java字段和ElementCreatorFactory，让两者可以独立变化。</p><p>配置文件的结构非常简洁：</p><div class="codeBlockContainer_Ckt0 theme-code-block" style="--prism-color:#393A34;--prism-background-color:#f6f8fa"><div class="codeBlockContent_biex"><pre tabindex="0" class="prism-code language-text codeBlock_bY9V thin-scrollbar"><code class="codeBlockLines_e6Vv"><span class="token-line" style="color:#393A34"><span class="token plain">{</span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">  "字段名": {</span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">    "label": "显示标签",</span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">    "elementCreator": "Factory类的全限定名",</span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">    "enum": "获取初始数据的方法",</span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">    "viewtype": "视图类型",</span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">    "help": "帮助文本"</span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">  }</span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">}</span><br></span></code></pre><div class="buttonGroup__atx"><button type="button" aria-label="Copy code to clipboard" title="Copy" class="clean-btn"><span class="copyButtonIcons_eSgA" aria-hidden="true"><svg class="copyButtonIcon_y97N" viewBox="0 0 24 24"><path d="M19,21H8V7H19M19,5H8A2,2 0 0,0 6,7V21A2,2 0 0,0 8,23H19A2,2 0 0,0 21,21V7A2,2 0 0,0 19,5M16,1H4A2,2 0 0,0 2,3V17H4V3H16V1Z"></path></svg><svg class="copyButtonSuccessIcon_LjdS" viewBox="0 0 24 24"><path d="M21,7L9,19L3.5,13.5L4.91,12.09L9,16.17L19.59,5.59L21,7Z"></path></svg></span></button></div></div></div><p>这种配置驱动的设计带来了几个好处：</p><ol><li><strong>热插拔</strong>：修改配置文件即可改变行为，无需重新编译</li><li><strong>可视化</strong>：配置文件清晰地展示了字段和Factory的对应关系</li><li><strong>解耦</strong>：Java代码和Factory类之间没有直接依赖</li></ol><p>更重要的是，这种设计让非开发人员也能理解系统的配置逻辑。当需要调整字段顺序、修改标签文本时，只需要编辑JSON文件即可。</p><h3 class="anchor anchorWithStickyNavbar_LWe7" id="组件化的可复用性">组件化的可复用性<a class="hash-link" href="#组件化的可复用性" title="Direct link to heading">​</a></h3><p>前端的组件化设计让UI的复用变得非常简单。</p><p>每个组件都是自包含的，它有自己的模板、样式和逻辑。当需要创建一个新的配置步骤时，可以参考现有组件的实现，快速搭建出新的UI。</p><p>组件之间通过标准的Input和Output进行通信：</p><ul><li>Input接收父组件传递的数据和配置</li><li>Output向父组件发送事件和数据变更</li></ul><p>这种标准化的通信方式，让组件可以像乐高积木一样自由组合。比如，一个下拉选择组件可以在多个不同的配置步骤中使用，只需要传入不同的选项数据即可。</p><p>Angular的依赖注入机制进一步增强了组件的可测试性。每个组件可以独立进行单元测试，不需要启动整个应用。</p><h2 class="anchor anchorWithStickyNavbar_LWe7" id="实际应用主表与维表join配置">实际应用：主表与维表JOIN配置<a class="hash-link" href="#实际应用主表与维表join配置" title="Direct link to heading">​</a></h2><p>让我们通过一个实际的应用场景，来看看这套架构是如何工作的。</p><h3 class="anchor anchorWithStickyNavbar_LWe7" id="join配置的业务场景">JOIN配置的业务场景<a class="hash-link" href="#join配置的业务场景" title="Direct link to heading">​</a></h3><p>在数据集成场景中，经常需要将主表和维表进行JOIN操作，以实现数据的宽表化。比如订单表需要关联用户表，获取用户的详细信息；商品表需要关联分类表，获取分类名称等。</p><p>这个配置过程涉及多个步骤：</p><ol><li>选择维表的数据源（可能有多个数据库）</li><li>选择具体的维表</li><li>设置JOIN的匹配条件（主表的哪个字段等于维表的哪个字段）</li><li>设置过滤条件（可选，比如只关联有效的记录）</li><li>选择需要输出的列</li><li>配置缓存策略</li></ol><p>如果用传统的单页面方式，这个表单会非常复杂。而使用多步配置插件，可以将这个过程分解为三个清晰的步骤。</p><p><strong>【图片位置3：JOIN配置三步向导界面截图】</strong>
<em>说明：展示三个步骤的界面，第一步选择数据源，第二步选择表，第三步配置匹配条件、过滤条件和输出列</em></p><h3 class="anchor anchorWithStickyNavbar_LWe7" id="三步配置的设计思路">三步配置的设计思路<a class="hash-link" href="#三步配置的设计思路" title="Direct link to heading">​</a></h3><p><strong>第一步：选择数据源</strong></p><p>这一步的职责非常明确：让用户从已配置的数据源中选择一个。实现上，创建一个JoinerSelectDataSource类，继承OneStepOfMultiSteps，定义一个dbName字段用于存储选择的数据源名称。</p><p>前端对应一个简单的下拉选择组件，选项从后端的数据源列表中获取。</p><p><strong>第二步：选择目标表</strong></p><p>这一步依赖第一步的选择结果。JoinerSelectTable类可以通过getOneStepOf方法获取第一步的配置，从而知道用户选择了哪个数据源，进而加载该数据源下的表列表。</p><p>这种步骤间的数据传递机制，让配置流程具有了"智能"——后续步骤可以根据前面的选择动态调整。</p><p><strong>第三步：配置匹配条件和输出列</strong></p><p>这是最复杂的一步，需要配置多个匹配条件、多个过滤条件，还要选择输出列。</p><p>JoinerSetMatchConditionAndCols类定义了多个字段：</p><ul><li>matchCondition：匹配条件列表</li><li>filterConditions：过滤条件列表</li><li>targetCols：输出列列表</li><li>colPrefix：列名前缀</li><li>cache：缓存配置</li></ul><p>每个字段都通过@FormField注解标注，指定了字段类型、验证规则等。关键的是，matchCondition和filterConditions字段的类型是MULTI_SELECTABLE，表示这是一个复杂的多元组配置。</p><h3 class="anchor anchorWithStickyNavbar_LWe7" id="过滤条件功能的扩展实现">过滤条件功能的扩展实现<a class="hash-link" href="#过滤条件功能的扩展实现" title="Direct link to heading">​</a></h3><p>最初的JOIN配置只支持匹配条件，后来需要新增过滤条件功能。这个扩展过程完美展示了架构的灵活性。</p><p><strong>后端扩展</strong>：</p><ol><li>创建TableJoinFilterCondition类，定义5个字段：tableType（主表/维表）、columnName（列名）、operator（运算符）、valueType（值类型）、value（值）</li><li>创建TableJoinFilterConditionCreatorFactory类，实现数据解析逻辑，并提供运算符、值类型等元数据</li><li>在ViewContent枚举中添加TableJoinFilterCondition值</li><li>在JoinerSetMatchConditionAndCols类中添加filterConditions字段</li><li>在JSON配置文件中添加filterConditions的配置项</li></ol><p><strong>前端扩展</strong>：</p><ol><li>创建TableJoinFilterConditionComponent组件，渲染过滤条件的表单</li><li>在TuplesPropertyType枚举中添加TableJoinFilterCondition值</li><li>在类型映射的switch语句中添加对应的case分支</li><li>在Angular模块中声明新组件</li></ol><p>整个扩展过程，没有修改任何现有的代码，只是新增了文件和配置。这就是良好架构设计带来的红利。</p><h2 class="anchor anchorWithStickyNavbar_LWe7" id="技术优势总结">技术优势总结<a class="hash-link" href="#技术优势总结" title="Direct link to heading">​</a></h2><h3 class="anchor anchorWithStickyNavbar_LWe7" id="开发效率的显著提升">开发效率的显著提升<a class="hash-link" href="#开发效率的显著提升" title="Direct link to heading">​</a></h3><p>通过这套架构，新增一个多步配置功能的开发成本大幅降低。</p><p>传统方式下，开发一个复杂的配置功能可能需要：</p><ul><li>设计数据库表结构</li><li>编写后端的CRUD接口</li><li>设计前端的表单布局</li><li>实现前后端的数据交互</li><li>编写验证逻辑</li><li>处理各种边界情况</li></ul><p>整个过程可能需要几天甚至一周的时间。</p><p>而使用多步配置插件架构，开发者只需要：</p><ul><li>定义数据模型（实现接口）</li><li>实现Factory类（套用模板）</li><li>编写前端组件（参考现有组件）</li><li>添加配置项（编辑JSON文件）</li></ul><p>熟练的开发者可以在半天内完成一个新功能的开发。这种效率的提升，来自于架构的标准化和模板化。</p><h3 class="anchor anchorWithStickyNavbar_LWe7" id="用户体验的优化">用户体验的优化<a class="hash-link" href="#用户体验的优化" title="Direct link to heading">​</a></h3><p>向导式的配置流程，让用户不再面对复杂的表单，而是一步步地完成配置。每一步的目标都很明确，用户不会感到迷茫。</p><p>步骤之间的数据联动，让配置过程更加智能。比如选择了数据源后，表列表会自动加载；选择了表后，列信息会自动展示。用户不需要手动刷新或重新加载。</p><p>实时的验证反馈，让用户在填写过程中就能发现错误，而不是提交后才被告知配置有误。这种即时反馈大大提升了配置的成功率。</p><h3 class="anchor anchorWithStickyNavbar_LWe7" id="系统可维护性的增强">系统可维护性的增强<a class="hash-link" href="#系统可维护性的增强" title="Direct link to heading">​</a></h3><p>清晰的架构边界，让代码的职责非常明确。当需要修改某个功能时，开发者可以快速定位到对应的类和组件，不会影响其他功能。</p><p>接口驱动的设计，让系统具有很好的可测试性。每个接口的实现都可以独立测试，不需要启动整个系统。这让单元测试变得简单可行。</p><p>配置文件的使用，让系统的行为更加透明。运维人员可以通过查看配置文件，了解系统的配置逻辑，而不需要阅读代码。</p><h2 class="anchor anchorWithStickyNavbar_LWe7" id="扩展场景展望">扩展场景展望<a class="hash-link" href="#扩展场景展望" title="Direct link to heading">​</a></h2><p>这套多步配置插件架构的应用场景远不止JOIN配置。它可以应用于任何需要复杂配置的场景。</p><p><strong>数据转换规则配置</strong>：数据同步过程中，经常需要对数据进行转换，比如字段映射、类型转换、格式化等。这些转换规则的配置可以分为多个步骤：选择源字段、选择转换函数、配置参数、预览结果。</p><p><strong>复杂数据源配置</strong>：某些数据源的配置非常复杂，比如Kafka需要配置集群地址、Topic、消费组、序列化方式等。通过多步配置，可以将这些配置项分组，让用户逐步完成。</p><p><strong>工作流编排配置</strong>：数据处理流程可能包含多个节点，每个节点有自己的配置。通过多步配置，可以让用户先定义流程结构，再逐个配置节点参数。</p><p><strong>机器学习模型配置</strong>：训练一个机器学习模型需要配置数据源、特征工程、算法参数、评估指标等。这个过程天然适合用多步配置来实现。</p><p>这些场景的共同特点是：配置项多、步骤间有依赖、需要向导式的用户体验。只要符合这些特点，都可以使用这套架构来实现。</p><h2 class="anchor anchorWithStickyNavbar_LWe7" id="总结">总结<a class="hash-link" href="#总结" title="Direct link to heading">​</a></h2><p>TIS的多步配置插件架构，通过精心设计的接口体系、配置驱动的灵活机制、组件化的前端实现，构建了一个既强大又易于扩展的配置系统。</p><p>这套架构的核心思想可以总结为几点：</p><ol><li><strong>接口抽象</strong>：通过接口定义标准流程，让扩展变得简单</li><li><strong>配置驱动</strong>：用配置文件连接各个组件，实现解耦</li><li><strong>组件化</strong>：前后端都采用组件化设计，提高复用性</li><li><strong>类型安全</strong>：通过枚举类型实现前后端的类型映射，减少错误</li></ol><p>对于开发者来说，这套架构提供了一个很好的参考：当面对复杂的配置场景时，不要试图用一个大而全的表单来解决，而是应该思考如何将配置过程分解为多个步骤，如何设计接口让系统具有扩展性，如何通过配置来驱动行为。</p><p>良好的架构设计，不仅能提升开发效率，更能让系统具有持续演进的能力。TIS的多步配置插件架构，正是这样一个经过实践检验的优秀设计。</p><hr><p><em>本文基于TIS（Table Integration System）开源项目的实际开发经验总结而成。TIS是一个企业级的数据集成平台，致力于为数据工程师提供高效、易用的数据同步和处理工具。</em></p>]]></content>
    </entry>
    <entry>
        <title type="html"><![CDATA[TIS平台上线智能监控功能：让Flink任务异常无处遁形]]></title>
        <id>flink-job-status-monitor</id>
        <link href="https://tis.pub/blog/flink-job-status-monitor"/>
        <updated>2025-11-17T00:00:00.000Z</updated>
        <summary type="html"><![CDATA[引言]]></summary>
        <content type="html"><![CDATA[<h2 class="anchor anchorWithStickyNavbar_LWe7" id="引言">引言<a class="hash-link" href="#引言" title="Direct link to heading">​</a></h2><p>在数字化转型的今天,企业越来越依赖实时数据处理来支撑业务决策。然而,实时任务一旦出现故障,可能导致数据丢失、业务中断等严重后果。传统的做法是安排专人盯盘监控,这不仅效率低下,而且无法做到24小时不间断监控。</p><p>TIS平台最新上线的<strong>Flink任务智能监控与报警系统</strong>,彻底解决了这一痛点。系统能够自动发现任务异常,并第一时间通过钉钉、企业微信、邮件等多种方式通知相关人员，真正做到"任务有问题，立即能知道"。</p><p>以下是钉钉群中接收到一条TIS发送的报警消息：</p><figure><a data-fancybox="true" href="/assets/images/alert-dingding-example-643671f77741959282d99c6ebd40e0eb.png"><img src="/assets/images/alert-dingding-example-643671f77741959282d99c6ebd40e0eb.png"></a></figure><h2 class="anchor anchorWithStickyNavbar_LWe7" id="传统监控方式的痛点">传统监控方式的痛点<a class="hash-link" href="#传统监控方式的痛点" title="Direct link to heading">​</a></h2><p>在引入自动化监控之前,企业通常采用以下方式来监控Flink任务:</p><h3 class="anchor anchorWithStickyNavbar_LWe7" id="人工定期检查">人工定期检查<a class="hash-link" href="#人工定期检查" title="Direct link to heading">​</a></h3><p>运维人员需要定期登录Flink管理界面,逐个检查任务状态。这种方式存在明显问题:</p><ul><li><strong>响应滞后</strong>: 任务失败后可能几小时才被发现</li><li><strong>人力成本高</strong>: 需要专人盯盘,占用大量人力</li><li><strong>覆盖不全</strong>: 夜间、周末、节假日可能无人值守</li><li><strong>容易遗漏</strong>: 任务数量多时,容易漏检</li></ul><h3 class="anchor anchorWithStickyNavbar_LWe7" id="简单的脚本监控">简单的脚本监控<a class="hash-link" href="#简单的脚本监控" title="Direct link to heading">​</a></h3><p>一些技术团队会编写简单的Shell脚本定时检查任务状态,但这种方式也有局限:</p><ul><li><strong>通知渠道单一</strong>: 通常只能发邮件,不够及时</li><li><strong>维护成本高</strong>: 每增加一个任务就要修改脚本</li><li><strong>缺乏统一管理</strong>: 脚本分散在各处,难以维护</li><li><strong>报警信息简陋</strong>: 只能提供基本的状态信息,缺少详细数据</li></ul><h3 class="anchor anchorWithStickyNavbar_LWe7" id="第三方监控工具">第三方监控工具<a class="hash-link" href="#第三方监控工具" title="Direct link to heading">​</a></h3><p>使用Prometheus、Grafana等监控工具,需要:</p><ul><li><strong>学习成本高</strong>: 需要掌握新的技术栈</li><li><strong>配置复杂</strong>: 需要编写大量配置文件</li><li><strong>集成困难</strong>: 与现有系统集成需要开发工作</li><li><strong>额外成本</strong>: 需要维护额外的监控系统</li></ul><h2 class="anchor anchorWithStickyNavbar_LWe7" id="tis智能监控方案的优势">TIS智能监控方案的优势<a class="hash-link" href="#tis智能监控方案的优势" title="Direct link to heading">​</a></h2><p>面对上述痛点,TIS平台推出了开箱即用的智能监控方案,具有以下显著优势:</p><h3 class="anchor anchorWithStickyNavbar_LWe7" id="1-零配置开箱即用">1. 零配置,开箱即用<a class="hash-link" href="#1-零配置开箱即用" title="Direct link to heading">​</a></h3><p>TIS的监控功能是平台内置的,无需安装额外的软件或服务。系统启动后,监控功能自动运行:</p><ul><li>✅ <strong>自动发现</strong>: 自动监控所有运行中的Flink任务,无需手动配置</li><li>✅ <strong>实时响应</strong>: 每5秒检测一次,任务异常后最快5秒即可发现</li><li>✅ <strong>智能判断</strong>: 区分正常停止和异常中断,避免误报</li></ul><p>以下是Flink任务监控系统工作流程：</p><figure><a data-fancybox="true" href="/assets/images/tis-flink-monitor-alert-workflow-e900df24fbe8d1b05466f0fea2740fda.jpg"><img src="/assets/images/tis-flink-monitor-alert-workflow-e900df24fbe8d1b05466f0fea2740fda.jpg"></a></figure><p><strong>流程说明</strong>:</p><ol><li><strong>监控启动</strong>: TIS平台启动时自动启动FlinkJobsMonitor监控器</li><li><strong>定时轮询</strong>: 每5秒执行一次任务状态检测</li><li><strong>状态检测</strong>: 获取所有Flink任务的当前状态</li><li><strong>状态比对</strong>: 与上次记录的状态进行对比,判断是否发生变化</li><li><strong>异常判断</strong>: 识别以下异常情况:<ul><li>任务失败 (RUNNING → FAILED)</li><li>任务丢失 (RUNNING → LOST)</li><li>异常取消 (RUNNING → CANCELED 且非手动停止)</li></ul></li><li><strong>报警触发</strong>: 构建包含任务详细信息的AlertTemplate对象</li><li><strong>多渠道发送</strong>: 并发发送到所有已配置的报警渠道</li><li><strong>模板渲染</strong>: 使用Velocity模板渲染个性化的报警消息</li><li><strong>消息发送</strong>: 根据不同渠道的协议发送报警通知</li><li><strong>状态更新</strong>: 更新任务状态记录,继续下一轮监控</li></ol><h3 class="anchor anchorWithStickyNavbar_LWe7" id="2-多渠道报警信息必达">2. 多渠道报警,信息必达<a class="hash-link" href="#2-多渠道报警信息必达" title="Direct link to heading">​</a></h3><p>TIS支持5种主流的报警渠道,满足不同场景的需求:</p><p><strong>📧 邮件报警</strong></p><ul><li>适合重要告警的归档和追溯</li><li>支持发送到多个邮箱</li><li>提供精美的HTML格式报告</li></ul><figure><a data-fancybox="true" href="/assets/images/EmailAlertChannel-c6d94971c1e2be30692596044964eb96.png"><img src="/assets/images/EmailAlertChannel-c6d94971c1e2be30692596044964eb96.png"></a></figure><p><strong>💬 钉钉报警</strong></p><ul><li>研发团队最常用的即时通讯工具</li><li>消息直达钉钉群,响应迅速</li><li>支持@指定人员,确保信息触达</li></ul><figure><a data-fancybox="true" href="/assets/images/DingTalkAlertChannel-13d37b31ad8e1987ee6dfd00a4001066.png"><img src="/assets/images/DingTalkAlertChannel-13d37b31ad8e1987ee6dfd00a4001066.png"></a></figure><p><strong>🏢 企业微信报警</strong></p><ul><li>适合使用企业微信的团队</li><li>支持Markdown格式,信息清晰</li><li>可@多人协同处理</li></ul><figure><a data-fancybox="true" href="/assets/images/WeComAlertChannel-f9373ec25483a591470c1b175264e29d.png"><img src="/assets/images/WeComAlertChannel-f9373ec25483a591470c1b175264e29d.png"></a></figure><p><strong>🚀 飞书报警</strong></p><ul><li>字节跳动旗下的协作平台</li><li>支持丰富的卡片样式</li><li>信息展示更加美观</li></ul><figure><a data-fancybox="true" href="/assets/images/LarkAlertChannel-1f27670d89cc6fbe9df2135e73d029d4.png"><img src="/assets/images/LarkAlertChannel-1f27670d89cc6fbe9df2135e73d029d4.png"></a></figure><p><strong>🔗 HTTP回调</strong></p><ul><li>最灵活的集成方式</li><li>可对接任何支持HTTP接口的系统</li><li>适合与现有运维系统集成</li></ul><figure><a data-fancybox="true" href="/assets/images/HttpCallbackAlertChannel-470042d486eedca9a5ee032117e3cb4e.png"><img src="/assets/images/HttpCallbackAlertChannel-470042d486eedca9a5ee032117e3cb4e.png"></a></figure><p><strong>核心优势</strong>: 可以同时配置多个渠道!例如:邮件用于归档,钉钉用于即时响应,企业微信通知运维团队,真正做到"多管齐下,确保必达"。</p><figure><a data-fancybox="true" href="/assets/images/alert-set-channel-dd309534dd892e0a790713b5a8ca30b1.png"><img src="/assets/images/alert-set-channel-dd309534dd892e0a790713b5a8ca30b1.png"></a></figure><h3 class="anchor anchorWithStickyNavbar_LWe7" id="3-报警信息丰富一目了然">3. 报警信息丰富,一目了然<a class="hash-link" href="#3-报警信息丰富一目了然" title="Direct link to heading">​</a></h3><p>TIS的报警消息不是简单的"任务失败"几个字,而是提供了全面的任务信息:</p><p><strong>基础信息</strong></p><ul><li>任务名称和状态(失败/丢失/异常取消)</li><li>开始时间、结束时间、运行时长</li><li>一键跳转到Flink Web UI查看详情</li></ul><p><strong>智能分析</strong></p><ul><li>自动计算运行时长</li><li>标注异常类型(首次失败/重复失败)</li><li>提供历史状态对比</li></ul><p>以下是通过邮件渠道接收到一封TIS报警消息：</p><figure><a data-fancybox="true" href="/assets/images/alert-email-example-0173d7050ff1bf00638beef7c77b00e7.png"><img src="/assets/images/alert-email-example-0173d7050ff1bf00638beef7c77b00e7.png"></a></figure><p><strong>举个例子</strong>: 凌晨3点,某个数据同步任务突然失败。5秒后,值班人员的钉钉群就收到了报警消息,上面清楚地显示:</p><ul><li>哪个任务出问题了</li><li>什么时候出的问题</li><li>已经运行了多长时间</li><li>点击链接可以直接查看Flink控制台
值班人员不用登录系统,在手机上就能了解全部情况,快速做出响应。</li></ul><h3 class="anchor anchorWithStickyNavbar_LWe7" id="4-灵活定制满足个性化需求">4. 灵活定制,满足个性化需求<a class="hash-link" href="#4-灵活定制满足个性化需求" title="Direct link to heading">​</a></h3><p>虽然TIS提供了开箱即用的默认配置,但也充分考虑了不同企业的个性化需求:</p><p><strong>自定义报警内容</strong></p><ul><li>支持Velocity模板,可以自定义报警消息的格式和内容</li><li>可以添加企业特有的信息(如项目名称、负责人联系方式等)</li><li>不同渠道可以使用不同的消息格式</li></ul><p><strong>灵活的配置管理</strong></p><ul><li>Web界面配置,无需修改代码</li><li>支持配置多个同类型渠道(如:生产环境钉钉群、测试环境钉钉群)</li><li>配置立即生效,无需重启服务</li></ul><p><strong>安全机制</strong></p><ul><li>钉钉和飞书支持密钥签名,防止Webhook泄露</li><li>邮件支持SSL加密传输</li><li>HTTP回调支持自定义请求头,可配置认证Token</li></ul><h3 class="anchor anchorWithStickyNavbar_LWe7" id="5-稳定可靠久经考验">5. 稳定可靠,久经考验<a class="hash-link" href="#5-稳定可靠久经考验" title="Direct link to heading">​</a></h3><p>TIS的监控系统在设计上充分考虑了稳定性和可靠性:</p><p><strong>异常隔离</strong></p><ul><li>单个任务检测失败不影响其他任务</li><li>某个报警渠道发送失败不影响其他渠道</li><li>监控系统异常不影响Flink任务本身的运行</li></ul><p><strong>性能优化</strong></p><ul><li>轻量级设计,对系统性能影响极小</li><li>高效的状态管理,避免重复报警</li><li>合理的轮询频率,在实时性和性能间取得平衡</li></ul><p><strong>久经考验</strong></p><ul><li>设计理念借鉴了Apache StreamPark等成熟开源项目</li><li>采用TIS平台久经考验的插件化架构</li><li>已在多个生产环境稳定运行</li></ul><figure><a data-fancybox="true" href="/assets/images/alert-art-diagram-e1c8d2ad2bf1c60ad9c58348e4406026.png"><img src="/assets/images/alert-art-diagram-e1c8d2ad2bf1c60ad9c58348e4406026.png"></a></figure><h2 class="anchor anchorWithStickyNavbar_LWe7" id="真实使用场景">真实使用场景<a class="hash-link" href="#真实使用场景" title="Direct link to heading">​</a></h2><h3 class="anchor anchorWithStickyNavbar_LWe7" id="场景一数据库增量同步任务监控">场景一:数据库增量同步任务监控<a class="hash-link" href="#场景一数据库增量同步任务监控" title="Direct link to heading">​</a></h3><p>某电商企业使用TIS搭建了MySQL到Elasticsearch的实时数据同步管道,用于实时更新商品搜索索引。</p><p><strong>面临的挑战</strong>:</p><ul><li>有20多个数据库表需要同步</li><li>任务需要7×24小时不间断运行</li><li>一旦中断会影响用户搜索体验</li></ul><p><strong>使用TIS监控后</strong>:</p><ul><li>配置了钉钉报警和邮件报警双通道</li><li>某次因为网络抖动导致一个同步任务失败</li><li>5秒内研发群收到钉钉报警,10分钟内问题得到解决</li><li>同时邮件报警提供了完整的事件记录,便于后续分析</li></ul><p><strong>效果</strong>:</p><ul><li>故障响应时间从平均2小时降低到10分钟</li><li>再也不用安排人员夜间值班</li><li>运维成本降低60%</li></ul><h3 class="anchor anchorWithStickyNavbar_LWe7" id="场景二大数据etl任务监控">场景二:大数据ETL任务监控<a class="hash-link" href="#场景二大数据etl任务监控" title="Direct link to heading">​</a></h3><p>某金融企业使用TIS处理每日的交易数据,将数据从业务库同步到数据仓库。</p><p><strong>面临的挑战</strong>:</p><ul><li>涉及多个业务系统,数据量大</li><li>对数据时效性要求高</li><li>需要及时发现并处理异常</li></ul><p><strong>使用TIS监控后</strong>:</p><ul><li>配置了企业微信报警,消息直达运维群</li><li>可以@不同的负责人处理不同任务的异常</li><li>报警消息中包含详细的任务信息和性能指标</li><li>通过HTTP回调将报警信息集成到公司的运维平台</li></ul><p><strong>效果</strong>:</p><ul><li>异常任务发现率100%</li><li>平均处理时间减少70%</li><li>建立了完整的运维响应机制</li></ul><h2 class="anchor anchorWithStickyNavbar_LWe7" id="如何使用tis监控功能">如何使用TIS监控功能<a class="hash-link" href="#如何使用tis监控功能" title="Direct link to heading">​</a></h2><h3 class="anchor anchorWithStickyNavbar_LWe7" id="第一步配置报警渠道">第一步:配置报警渠道<a class="hash-link" href="#第一步配置报警渠道" title="Direct link to heading">​</a></h3><p>以配置钉钉报警为例,整个过程只需3分钟:</p><p><strong>1. 创建钉钉群机器人</strong></p><ul><li>打开需要接收报警的钉钉群</li><li>点击"群设置" → "智能群助手" → "添加机器人" → "自定义"</li><li>设置机器人名称,安全设置选择"加签"</li><li>复制Webhook URL和密钥</li></ul><p><strong>2. 在TIS中配置报警渠道</strong></p><ul><li>登录TIS平台</li><li>进入"全局配置" → "报警渠道"</li><li>点击"添加",选择"DingTalk"</li><li>填写配置:<ul><li>渠道名称:如"生产环境钉钉"</li><li>Webhook URL:粘贴刚才复制的URL</li><li>密钥:粘贴刚才复制的密钥</li><li>(可选)填写需要@的人员手机号</li></ul></li><li>保存配置</li></ul><p><strong>3. 完成</strong></p><ul><li>配置保存后立即生效</li><li>系统自动开始监控所有Flink任务</li><li>有异常时自动发送报警到钉钉群</li></ul><h3 class="anchor anchorWithStickyNavbar_LWe7" id="第二步测试报警功能">第二步:测试报警功能<a class="hash-link" href="#第二步测试报警功能" title="Direct link to heading">​</a></h3><p>配置完成后,建议先测试一下:</p><ol><li>在TIS中找一个测试任务</li><li>手动停止该任务</li><li>等待5-10秒</li><li>检查钉钉群是否收到报警消息</li></ol><p>收到报警说明配置成功!消息中会显示:</p><ul><li>任务名称和状态</li><li>运行时长</li><li>查看详情的链接</li></ul><h3 class="anchor anchorWithStickyNavbar_LWe7" id="第三步配置多个渠道可选">第三步:配置多个渠道(可选)<a class="hash-link" href="#第三步配置多个渠道可选" title="Direct link to heading">​</a></h3><p>根据需要,您还可以配置其他报警渠道:</p><ul><li><strong>邮件</strong>: 用于重要告警的归档,方便后续查询</li><li><strong>企业微信</strong>: 通知运维团队</li><li><strong>飞书</strong>: 如果团队使用飞书办公</li><li><strong>HTTP回调</strong>: 集成到现有的运维系统</li></ul><p>每个渠道的配置方式类似,都是在"报警渠道"页面点击"添加",填写相应的配置信息即可。</p><h2 class="anchor anchorWithStickyNavbar_LWe7" id="核心价值总结">核心价值总结<a class="hash-link" href="#核心价值总结" title="Direct link to heading">​</a></h2><p>TIS Flink任务智能监控系统为企业带来的核心价值:</p><h3 class="anchor anchorWithStickyNavbar_LWe7" id="-降低成本">💰 降低成本<a class="hash-link" href="#-降低成本" title="Direct link to heading">​</a></h3><ul><li>不需要专人盯盘,节省人力成本</li><li>无需部署额外的监控系统,降低IT成本</li><li>快速响应减少故障损失</li></ul><h3 class="anchor anchorWithStickyNavbar_LWe7" id="-提升效率">⚡ 提升效率<a class="hash-link" href="#-提升效率" title="Direct link to heading">​</a></h3><ul><li>5秒发现异常,响应速度提升数十倍</li><li>自动化监控,释放运维人力</li><li>多渠道通知,确保信息必达</li></ul><h3 class="anchor anchorWithStickyNavbar_LWe7" id="️-保障稳定">🛡️ 保障稳定<a class="hash-link" href="#️-保障稳定" title="Direct link to heading">​</a></h3><ul><li>7×24小时不间断监控</li><li>智能判断,避免误报和漏报</li><li>详细信息,快速定位问题</li></ul><h3 class="anchor anchorWithStickyNavbar_LWe7" id="-易于管理">📊 易于管理<a class="hash-link" href="#-易于管理" title="Direct link to heading">​</a></h3><ul><li>Web界面配置,无需技术背景</li><li>统一管理所有Flink任务</li><li>支持多团队、多项目使用</li></ul><h3 class="anchor anchorWithStickyNavbar_LWe7" id="-灵活扩展">🔧 灵活扩展<a class="hash-link" href="#-灵活扩展" title="Direct link to heading">​</a></h3><ul><li>5种主流报警渠道,满足不同需求</li><li>支持自定义报警模板</li><li>可集成到现有运维体系</li></ul><h2 class="anchor anchorWithStickyNavbar_LWe7" id="技术实现亮点">技术实现亮点<a class="hash-link" href="#技术实现亮点" title="Direct link to heading">​</a></h2><p>虽然本文面向非技术人员,但有必要简单介绍一下TIS监控系统的技术优势,这些设计保证了系统的稳定可靠:</p><h3 class="anchor anchorWithStickyNavbar_LWe7" id="借鉴成熟开源项目">借鉴成熟开源项目<a class="hash-link" href="#借鉴成熟开源项目" title="Direct link to heading">​</a></h3><ul><li>参考Apache StreamPark的监控设计理念</li><li>站在开源社区的肩膀上,避免重复造轮车</li><li>结合TIS平台特点进行优化</li></ul><h3 class="anchor anchorWithStickyNavbar_LWe7" id="插件化架构">插件化架构<a class="hash-link" href="#插件化架构" title="Direct link to heading">​</a></h3><ul><li>新增报警渠道无需修改核心代码</li><li>各个报警渠道独立运行,互不影响</li><li>方便后续扩展更多渠道</li></ul><h3 class="anchor anchorWithStickyNavbar_LWe7" id="智能状态管理">智能状态管理<a class="hash-link" href="#智能状态管理" title="Direct link to heading">​</a></h3><ul><li>记录任务的历史状态,避免重复报警</li><li>自动清理过期数据,防止内存泄漏</li><li>高性能的状态比对算法</li></ul><h3 class="anchor anchorWithStickyNavbar_LWe7" id="模板化设计">模板化设计<a class="hash-link" href="#模板化设计" title="Direct link to heading">​</a></h3><ul><li>统一的消息模板变量体系</li><li>用户可以自由定制报警内容</li><li>同一套模板适用于所有渠道</li></ul><h3 class="anchor anchorWithStickyNavbar_LWe7" id="完善的安全机制">完善的安全机制<a class="hash-link" href="#完善的安全机制" title="Direct link to heading">​</a></h3><ul><li>支持密钥签名验证(钉钉、飞书)</li><li>支持SSL加密传输(邮件)</li><li>支持自定义认证(HTTP回调)</li></ul><h2 class="anchor anchorWithStickyNavbar_LWe7" id="结语">结语<a class="hash-link" href="#结语" title="Direct link to heading">​</a></h2><p>在数字化时代,数据就是企业的生命线。实时数据处理任务的稳定运行,直接关系到业务的正常开展。TIS Flink任务智能监控系统,让企业可以放心地运行大规模实时数据管道,再也不用担心任务异常无人知晓。</p><p><strong>开箱即用</strong>的设计理念、<strong>多渠道</strong>的报警支持、<strong>丰富详细</strong>的报警信息,让TIS的监控功能成为运维人员的得力助手。无论是中小企业还是大型互联网公司,都能从中受益。</p><p>如果您正在使用TIS平台,现在就可以开始配置报警渠道,让监控功能为您的Flink任务保驾护航。如果还没有使用TIS,欢迎访问我们的官网了解更多信息。</p>]]></content>
        <author>
            <name>百岁</name>
            <uri>https://github.com/baisui1981</uri>
        </author>
    </entry>
    <entry>
        <title type="html"><![CDATA[使用依赖反转原则实现TIS增量通道]]></title>
        <id>dip-in-tis</id>
        <link href="https://tis.pub/blog/dip-in-tis"/>
        <updated>2024-05-10T00:00:00.000Z</updated>
        <summary type="html"><![CDATA[前言]]></summary>
        <content type="html"><![CDATA[<h2 class="anchor anchorWithStickyNavbar_LWe7" id="前言">前言<a class="hash-link" href="#前言" title="Direct link to heading">​</a></h2><p>说到<strong>依赖反转原则</strong> ，<a href="https://baike.baidu.com/item/%E4%BE%9D%E8%B5%96%E5%8F%8D%E8%BD%AC%E5%8E%9F%E5%88%99/22718037?fr=ge_ala" target="_blank" rel="noopener noreferrer">百度百科是这样介绍的</a>，如下：</p><blockquote><p>在面向对象编程领域中，依赖反转原则（Dependency inversion principle，DIP）是指一种特定的解耦（传统的依赖关系创建在高层次上，而具体的策略设置则应用在低层次的模块上）形式，使得高层次的模块不依赖于低层次的模块的实现细节，依赖关系被颠倒（反转），从而使得低层次模块依赖于高层次模块的需求抽象。</p></blockquote><p>大家应该会马上会联想到Spring Framework，在介绍Spring Framework框架常常会提及依赖反转原则，看上面的介绍估计会云里雾里，说得通俗一点，该原则的初衷<strong>要求服务提供者与服务调用者在代码实现层面实现解耦</strong>。</p><p>为了加深理解，经常会提到的一个例子，以前古时候的包办婚姻，假如是男方到了适婚年龄，只要把自己的条件、要求告诉媒婆。接下来找合适对象的过程就交给媒婆就行了，男方只需要负责到时候入洞房就行了。对于男方来说，
婚姻和他是服务提供者和消费者的关系，由于引入了媒婆的角色，男方省去了谈婚论嫁的麻烦过程，只需要专注于核心业务-入洞房。最终使得整个过程显得简单高效，形式格外优雅。</p><p>因此，<strong>依赖反转原则DIP</strong>作为一个朴素的原则存在，可以应用到软件设计领域每一个流程环节当中，而不仅仅适用于Spring Framework当中。</p><p>本文就以<strong>依赖反转原则DIP</strong>在TIS增量实时数据通道的设计、实现过程中如何利用这一原则来优化设计、实现流程进行阐述。</p><h2 class="anchor anchorWithStickyNavbar_LWe7" id="实现实时增量数据管道需求">实现实时增量数据管道需求<a class="hash-link" href="#实现实时增量数据管道需求" title="Direct link to heading">​</a></h2><p>在TIS中为用户提供了基于Flink端到端的实时增量数据通道功能，市面上已经提供了基于Flink和<a href="https://github.com/apache/flink-cdc" target="_blank" rel="noopener noreferrer">Flink-CDC</a>的实时流同步工具，从用户反馈来看已经很方便了，那为什么还要通过TIS来
使用Flink-CDC呢？</p><p>这是一个非常好的问题，要回答这个问题，首先我们需要从用户的角度了解用户到底需要什么？然后从需求出发设计并且构建出用户体验达到极致的产品。</p><p>大数据流计算领域，用户的核心需求是：</p><ol><li>可追溯操作历史的控制系统，这样可以方便回滚历史操作。 </li><li>不关心算子实现细节，流计算的使用者往往是对Flink不了解的数据分析人员，所以在产品使用体验上需要屏蔽底层技术细节。</li><li>可扩展的端类型：Flink-CDC从3.0版本支持的<a href="https://nightlies.apache.org/flink/flink-cdc-docs-release-3.0/docs/connectors/overview/" target="_blank" rel="noopener noreferrer">Connectors</a>，只支持了有限个数的基于增量监听CDC技术的Source端
，和少量Sink端实现，如：Doris和StarRocks的Sink端类型。还远远没有达到用户实际生产场景下的端类型。所以，需要提供在更高层次上，通过便捷方式扩展Source和Sink端类型的手段。</li></ol><p>TIS正式为了弥补以上三个使用Flink-CDC框架中的不足而开发的。</p><h2 class="anchor anchorWithStickyNavbar_LWe7" id="具体实现">具体实现<a class="hash-link" href="#具体实现" title="Direct link to heading">​</a></h2><p>下面具体对以上第2点进行进行说明，配置并且触发执行基于Flink-CDC的数据管道具体通过以下步骤完成</p><iframe width="100%" height="500px" src="https://www.processon.com/embed/66407f1712931152f2850d1b?cid=66407f1712931152f2850d1e"></iframe><p><a href="https://www.processon.com/diagraming/66407f1712931152f2850d1b" target="_blank" rel="noopener noreferrer">编辑</a></p><p>在<code>构建DataStreamSource</code>步骤中，通过调用Flink-CDC提供的API代码，可以方便订阅到如MySQL的增量更新消息，如下代码：</p><div class="language-java codeBlockContainer_Ckt0 theme-code-block" style="--prism-color:#393A34;--prism-background-color:#f6f8fa"><div class="codeBlockContent_biex"><pre tabindex="0" class="prism-code language-java codeBlock_bY9V thin-scrollbar"><code class="codeBlockLines_e6Vv"><span class="token-line" style="color:#393A34"><span class="token keyword" style="color:#00009f">public</span><span class="token plain"> </span><span class="token keyword" style="color:#00009f">static</span><span class="token plain"> </span><span class="token keyword" style="color:#00009f">void</span><span class="token plain"> </span><span class="token function" style="color:#d73a49">main</span><span class="token punctuation" style="color:#393A34">(</span><span class="token class-name">String</span><span class="token punctuation" style="color:#393A34">[</span><span class="token punctuation" style="color:#393A34">]</span><span class="token plain"> args</span><span class="token punctuation" style="color:#393A34">)</span><span class="token plain"> </span><span class="token keyword" style="color:#00009f">throws</span><span class="token plain"> </span><span class="token class-name">Exception</span><span class="token plain"> </span><span class="token punctuation" style="color:#393A34">{</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">    </span><span class="token class-name">MySqlSource</span><span class="token generics punctuation" style="color:#393A34">&lt;</span><span class="token generics class-name">String</span><span class="token generics punctuation" style="color:#393A34">&gt;</span><span class="token plain"> mySqlSource </span><span class="token operator" style="color:#393A34">=</span><span class="token plain"> </span><span class="token class-name">MySqlSource</span><span class="token punctuation" style="color:#393A34">.</span><span class="token generics punctuation" style="color:#393A34">&lt;</span><span class="token generics class-name">String</span><span class="token generics punctuation" style="color:#393A34">&gt;</span><span class="token function" style="color:#d73a49">builder</span><span class="token punctuation" style="color:#393A34">(</span><span class="token punctuation" style="color:#393A34">)</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">        </span><span class="token punctuation" style="color:#393A34">.</span><span class="token function" style="color:#d73a49">hostname</span><span class="token punctuation" style="color:#393A34">(</span><span class="token string" style="color:#e3116c">"yourHostname"</span><span class="token punctuation" style="color:#393A34">)</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">        </span><span class="token punctuation" style="color:#393A34">.</span><span class="token function" style="color:#d73a49">port</span><span class="token punctuation" style="color:#393A34">(</span><span class="token plain">yourPort</span><span class="token punctuation" style="color:#393A34">)</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">        </span><span class="token punctuation" style="color:#393A34">.</span><span class="token function" style="color:#d73a49">databaseList</span><span class="token punctuation" style="color:#393A34">(</span><span class="token string" style="color:#e3116c">"yourDatabaseName"</span><span class="token punctuation" style="color:#393A34">)</span><span class="token plain"> </span><span class="token comment" style="color:#999988;font-style:italic">// set captured database, If you need to synchronize the whole database, Please set tableList to ".*".</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">        </span><span class="token punctuation" style="color:#393A34">.</span><span class="token function" style="color:#d73a49">tableList</span><span class="token punctuation" style="color:#393A34">(</span><span class="token string" style="color:#e3116c">"yourDatabaseName.yourTableName"</span><span class="token punctuation" style="color:#393A34">)</span><span class="token plain"> </span><span class="token comment" style="color:#999988;font-style:italic">// set captured table</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">        </span><span class="token punctuation" style="color:#393A34">.</span><span class="token function" style="color:#d73a49">username</span><span class="token punctuation" style="color:#393A34">(</span><span class="token string" style="color:#e3116c">"yourUsername"</span><span class="token punctuation" style="color:#393A34">)</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">        </span><span class="token punctuation" style="color:#393A34">.</span><span class="token function" style="color:#d73a49">password</span><span class="token punctuation" style="color:#393A34">(</span><span class="token string" style="color:#e3116c">"yourPassword"</span><span class="token punctuation" style="color:#393A34">)</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">        </span><span class="token punctuation" style="color:#393A34">.</span><span class="token function" style="color:#d73a49">deserializer</span><span class="token punctuation" style="color:#393A34">(</span><span class="token keyword" style="color:#00009f">new</span><span class="token plain"> </span><span class="token class-name">JsonDebeziumDeserializationSchema</span><span class="token punctuation" style="color:#393A34">(</span><span class="token punctuation" style="color:#393A34">)</span><span class="token punctuation" style="color:#393A34">)</span><span class="token plain"> </span><span class="token comment" style="color:#999988;font-style:italic">// converts SourceRecord to JSON String</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">        </span><span class="token punctuation" style="color:#393A34">.</span><span class="token function" style="color:#d73a49">build</span><span class="token punctuation" style="color:#393A34">(</span><span class="token punctuation" style="color:#393A34">)</span><span class="token punctuation" style="color:#393A34">;</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain" style="display:inline-block"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">    </span><span class="token class-name">StreamExecutionEnvironment</span><span class="token plain"> env </span><span class="token operator" style="color:#393A34">=</span><span class="token plain"> </span><span class="token class-name">StreamExecutionEnvironment</span><span class="token punctuation" style="color:#393A34">.</span><span class="token function" style="color:#d73a49">getExecutionEnvironment</span><span class="token punctuation" style="color:#393A34">(</span><span class="token punctuation" style="color:#393A34">)</span><span class="token punctuation" style="color:#393A34">;</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain" style="display:inline-block"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">    </span><span class="token comment" style="color:#999988;font-style:italic">// enable checkpoint</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">    env</span><span class="token punctuation" style="color:#393A34">.</span><span class="token function" style="color:#d73a49">enableCheckpointing</span><span class="token punctuation" style="color:#393A34">(</span><span class="token number" style="color:#36acaa">3000</span><span class="token punctuation" style="color:#393A34">)</span><span class="token punctuation" style="color:#393A34">;</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain" style="display:inline-block"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">    env</span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">      </span><span class="token punctuation" style="color:#393A34">.</span><span class="token function" style="color:#d73a49">fromSource</span><span class="token punctuation" style="color:#393A34">(</span><span class="token plain">mySqlSource</span><span class="token punctuation" style="color:#393A34">,</span><span class="token plain"> </span><span class="token class-name">WatermarkStrategy</span><span class="token punctuation" style="color:#393A34">.</span><span class="token function" style="color:#d73a49">noWatermarks</span><span class="token punctuation" style="color:#393A34">(</span><span class="token punctuation" style="color:#393A34">)</span><span class="token punctuation" style="color:#393A34">,</span><span class="token plain"> </span><span class="token string" style="color:#e3116c">"MySQL Source"</span><span class="token punctuation" style="color:#393A34">)</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">      </span><span class="token comment" style="color:#999988;font-style:italic">// set 4 parallel source tasks</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">      </span><span class="token punctuation" style="color:#393A34">.</span><span class="token function" style="color:#d73a49">setParallelism</span><span class="token punctuation" style="color:#393A34">(</span><span class="token number" style="color:#36acaa">4</span><span class="token punctuation" style="color:#393A34">)</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">      </span><span class="token punctuation" style="color:#393A34">.</span><span class="token function" style="color:#d73a49">print</span><span class="token punctuation" style="color:#393A34">(</span><span class="token punctuation" style="color:#393A34">)</span><span class="token punctuation" style="color:#393A34">.</span><span class="token function" style="color:#d73a49">setParallelism</span><span class="token punctuation" style="color:#393A34">(</span><span class="token number" style="color:#36acaa">1</span><span class="token punctuation" style="color:#393A34">)</span><span class="token punctuation" style="color:#393A34">;</span><span class="token plain"> </span><span class="token comment" style="color:#999988;font-style:italic">// use parallelism 1 for sink to keep message ordering</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain" style="display:inline-block"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">    env</span><span class="token punctuation" style="color:#393A34">.</span><span class="token function" style="color:#d73a49">execute</span><span class="token punctuation" style="color:#393A34">(</span><span class="token string" style="color:#e3116c">"Print MySQL Snapshot + Binlog"</span><span class="token punctuation" style="color:#393A34">)</span><span class="token punctuation" style="color:#393A34">;</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">  </span><span class="token punctuation" style="color:#393A34">}</span><br></span></code></pre><div class="buttonGroup__atx"><button type="button" aria-label="Copy code to clipboard" title="Copy" class="clean-btn"><span class="copyButtonIcons_eSgA" aria-hidden="true"><svg class="copyButtonIcon_y97N" viewBox="0 0 24 24"><path d="M19,21H8V7H19M19,5H8A2,2 0 0,0 6,7V21A2,2 0 0,0 8,23H19A2,2 0 0,0 21,21V7A2,2 0 0,0 19,5M16,1H4A2,2 0 0,0 2,3V17H4V3H16V1Z"></path></svg><svg class="copyButtonSuccessIcon_LjdS" viewBox="0 0 24 24"><path d="M21,7L9,19L3.5,13.5L4.91,12.09L9,16.17L19.59,5.59L21,7Z"></path></svg></span></button></div></div></div><p>通过以上流计算的流程中可以使用创建出<code>MySqlSource&lt;String&gt; mySqlSource</code>Source加入到各种算子中去进行计算。</p><p>使用SQL的方式将Stream Source 注册为Flink Table：</p><div class="language-sql codeBlockContainer_Ckt0 theme-code-block" style="--prism-color:#393A34;--prism-background-color:#f6f8fa"><div class="codeBlockContent_biex"><pre tabindex="0" class="prism-code language-sql codeBlock_bY9V thin-scrollbar"><code class="codeBlockLines_e6Vv"><span class="token-line" style="color:#393A34"><span class="token keyword" style="color:#00009f">CREATE</span><span class="token plain"> </span><span class="token keyword" style="color:#00009f">TABLE</span><span class="token plain"> mysql_source </span><span class="token punctuation" style="color:#393A34">(</span><span class="token punctuation" style="color:#393A34">.</span><span class="token punctuation" style="color:#393A34">.</span><span class="token punctuation" style="color:#393A34">.</span><span class="token punctuation" style="color:#393A34">)</span><span class="token plain"> </span><span class="token keyword" style="color:#00009f">WITH</span><span class="token plain"> </span><span class="token punctuation" style="color:#393A34">(</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">    </span><span class="token string" style="color:#e3116c">'connector'</span><span class="token plain"> </span><span class="token operator" style="color:#393A34">=</span><span class="token plain"> </span><span class="token string" style="color:#e3116c">'mysql-cdc'</span><span class="token punctuation" style="color:#393A34">,</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">    </span><span class="token string" style="color:#e3116c">'scan.startup.mode'</span><span class="token plain"> </span><span class="token operator" style="color:#393A34">=</span><span class="token plain"> </span><span class="token string" style="color:#e3116c">'earliest-offset'</span><span class="token punctuation" style="color:#393A34">,</span><span class="token plain"> </span><span class="token comment" style="color:#999988;font-style:italic">-- Start from earliest offset</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">    </span><span class="token string" style="color:#e3116c">'scan.startup.mode'</span><span class="token plain"> </span><span class="token operator" style="color:#393A34">=</span><span class="token plain"> </span><span class="token string" style="color:#e3116c">'latest-offset'</span><span class="token punctuation" style="color:#393A34">,</span><span class="token plain"> </span><span class="token comment" style="color:#999988;font-style:italic">-- Start from latest offset</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">    </span><span class="token string" style="color:#e3116c">'scan.startup.mode'</span><span class="token plain"> </span><span class="token operator" style="color:#393A34">=</span><span class="token plain"> </span><span class="token string" style="color:#e3116c">'specific-offset'</span><span class="token punctuation" style="color:#393A34">,</span><span class="token plain"> </span><span class="token comment" style="color:#999988;font-style:italic">-- Start from specific offset</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">    </span><span class="token string" style="color:#e3116c">'scan.startup.mode'</span><span class="token plain"> </span><span class="token operator" style="color:#393A34">=</span><span class="token plain"> </span><span class="token string" style="color:#e3116c">'timestamp'</span><span class="token punctuation" style="color:#393A34">,</span><span class="token plain"> </span><span class="token comment" style="color:#999988;font-style:italic">-- Start from timestamp</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">    </span><span class="token string" style="color:#e3116c">'scan.startup.specific-offset.file'</span><span class="token plain"> </span><span class="token operator" style="color:#393A34">=</span><span class="token plain"> </span><span class="token string" style="color:#e3116c">'mysql-bin.000003'</span><span class="token punctuation" style="color:#393A34">,</span><span class="token plain"> </span><span class="token comment" style="color:#999988;font-style:italic">-- Binlog filename under specific offset startup mode</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">    </span><span class="token string" style="color:#e3116c">'scan.startup.specific-offset.pos'</span><span class="token plain"> </span><span class="token operator" style="color:#393A34">=</span><span class="token plain"> </span><span class="token string" style="color:#e3116c">'4'</span><span class="token punctuation" style="color:#393A34">,</span><span class="token plain"> </span><span class="token comment" style="color:#999988;font-style:italic">-- Binlog position under specific offset mode</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">    </span><span class="token string" style="color:#e3116c">'scan.startup.specific-offset.gtid-set'</span><span class="token plain"> </span><span class="token operator" style="color:#393A34">=</span><span class="token plain"> </span><span class="token string" style="color:#e3116c">'24DA167-0C0C-11E8-8442-00059A3C7B00:1-19'</span><span class="token punctuation" style="color:#393A34">,</span><span class="token plain"> </span><span class="token comment" style="color:#999988;font-style:italic">-- GTID set under specific offset startup mode</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">    </span><span class="token string" style="color:#e3116c">'scan.startup.timestamp-millis'</span><span class="token plain"> </span><span class="token operator" style="color:#393A34">=</span><span class="token plain"> </span><span class="token string" style="color:#e3116c">'1667232000000'</span><span class="token plain"> </span><span class="token comment" style="color:#999988;font-style:italic">-- Timestamp under timestamp startup mode</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">    </span><span class="token punctuation" style="color:#393A34">.</span><span class="token punctuation" style="color:#393A34">.</span><span class="token punctuation" style="color:#393A34">.</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain"></span><span class="token punctuation" style="color:#393A34">)</span><br></span></code></pre><div class="buttonGroup__atx"><button type="button" aria-label="Copy code to clipboard" title="Copy" class="clean-btn"><span class="copyButtonIcons_eSgA" aria-hidden="true"><svg class="copyButtonIcon_y97N" viewBox="0 0 24 24"><path d="M19,21H8V7H19M19,5H8A2,2 0 0,0 6,7V21A2,2 0 0,0 8,23H19A2,2 0 0,0 21,21V7A2,2 0 0,0 19,5M16,1H4A2,2 0 0,0 2,3V17H4V3H16V1Z"></path></svg><svg class="copyButtonSuccessIcon_LjdS" viewBox="0 0 24 24"><path d="M21,7L9,19L3.5,13.5L4.91,12.09L9,16.17L19.59,5.59L21,7Z"></path></svg></span></button></div></div></div><p>以上是Flink-CDC提供的标准化的Demo案例。</p><p>在这里我们重新用依赖反转原则来思考一下，这个构建流程是否有违背该原则？确实，从用户的角度来说，用户只关心最终构建出来的<code>MySqlSource&lt;String&gt;</code>实例，至于构建该实例的过程用户并不关心，所以在设计过程需要将
<code>MySqlSource&lt;String&gt;实例</code>构建过程与它的调用者之间进行解耦合。</p><p>是时候发挥TIS的作用了，TIS需要发挥实例容器的作用，由TIS根据用户配置的Source端参数自动地创建<code>MySqlSource&lt;String&gt;</code>实例，
在运行时自动注入到执行流程中。</p><ul><li>配置Source/Sink Connector<figure><a data-fancybox="true" href="/assets/images/source-sink-config-26a71b75918cfc0382e41c0e03db7cf7.png"><img src="/assets/images/source-sink-config-26a71b75918cfc0382e41c0e03db7cf7.png"></a></figure></li><li>直接引用TIS注入的SourceStream实例<figure><a data-fancybox="true" href="/assets/images/instance-by-tis-and-inject-850907bd03214cd7a11af8f07b5798fa.png"><img src="/assets/images/instance-by-tis-and-inject-850907bd03214cd7a11af8f07b5798fa.png"></a></figure></li><li>当用户选择Flink SQL类型脚本,直接引用已经注册完成的Table名即可<figure><a data-fancybox="true" href="/assets/images/instance-by-tis-and-register-flink-table-978a89528ef9c95afcf4b1b0a7f85369.png"><img src="/assets/images/instance-by-tis-and-register-flink-table-978a89528ef9c95afcf4b1b0a7f85369.png"></a></figure></li></ul><p>以上具体提供注入实例的封装工厂是：</p><ul><li>执行Stream Source创建： <a href="https://github.com/qlangtech/plugins/blob/master/tis-incr/tis-flink-cdc-mysql-plugin/src/main/java/com/qlangtech/tis/plugins/incr/flink/cdc/mysql/FlinkCDCMySQLSourceFactory.java" target="_blank" rel="noopener noreferrer">FlinkCDCMySQLSourceFactory.java</a></li><li>执行将Stream Source转成Flink Table： <a href="https://github.com/qlangtech/plugins/blob/master/tis-incr/tis-flink-chunjun-mysql-plugin/src/main/java/com/qlangtech/tis/plugins/incr/flink/connector/table/MySQLDynamicTableFactory.java" target="_blank" rel="noopener noreferrer">MySQLDynamicTableFactory.java</a></li></ul><p>以上两段代码的执行逻辑类似Spring <a href="https://www.baeldung.com/spring-factorybean" target="_blank" rel="noopener noreferrer">FactoryBean</a> 执行逻辑，实现容器预定的扩展工厂接口，运行期由容器负责初始化，继而将实例注入到需要反向依赖的实例中。</p><h2 class="anchor anchorWithStickyNavbar_LWe7" id="总结">总结<a class="hash-link" href="#总结" title="Direct link to heading">​</a></h2><p>本文介绍了利用依赖反转原则在TIS中实现实时增量通道的优化方法，可使最终用户最大限度地关注流式计算核心业务本身，其他的琐碎的与实例初始化相关的工作都交给TIS来完成即可。</p><p>与此类似的功能优化，在TIS实现过程中还有很多，会在日后的博客分享中陆续发表。</p>]]></content>
        <author>
            <name>百岁</name>
            <uri>https://github.com/baisui1981</uri>
        </author>
    </entry>
    <entry>
        <title type="html"><![CDATA[优化从MongoDB表中读取数据]]></title>
        <id>read-from-mongodb</id>
        <link href="https://tis.pub/blog/read-from-mongodb"/>
        <updated>2024-05-10T00:00:00.000Z</updated>
        <summary type="html"><![CDATA[前言]]></summary>
        <content type="html"><![CDATA[<h2 class="anchor anchorWithStickyNavbar_LWe7" id="前言">前言<a class="hash-link" href="#前言" title="Direct link to heading">​</a></h2><p>MongoDB是一个基于分布式文件存储的开源数据库系统，其内容存储格式为BSON（一种类json的二进制形式），其主要特点是高性能、易部署、易使用，存储数据支持高度事务性且支持完全索引，包括地理空间索引、散列索引和全文索引，还有一个比较大的优势是其适应于海量数据的存储，其数据被分散在不同的服务器上，以自动分区数据。</p><p>MongoDB在实际生产环境中有很多应用场景，利用器BSON数据结构可以在运行期动态扩展数据Schema。</p><p>MongoDB被TIS整合进了数据集成方案，通过TIS中可以<strong>方便对MongoDB端进行读取或者写入操作</strong>，方便实现对MongoDB的数据迁移、实时容灾备份、异构数据端（如：Doris）实时同步实现复杂OLAP操作。</p><p>本文就实际操作过程中，发现从MongoDB中不能预先读取表Schema，对此进行了优化，并且对此优化过程作以详细介绍。</p><h2 class="anchor anchorWithStickyNavbar_LWe7" id="发现瓶颈">发现瓶颈<a class="hash-link" href="#发现瓶颈" title="Direct link to heading">​</a></h2><p>MongoDB作为文档型数据库的代表，区别于传统关系型数据库MySQL，MongoDB的表数据结构Schema在运行期是可变的，而不是像MySQL那样通过Create Table DDL预先定义好表Schema。因此，在为MongoDB做数据集成操作时带来一个麻烦事儿，需要通过手工配置的方式
为读取MongoDB的表作为依据。例如，用户通过Alibaba DataX来读取MongoDB需编写DataX Reader任务配置：</p><div class="language-json codeBlockContainer_Ckt0 theme-code-block" style="--prism-color:#393A34;--prism-background-color:#f6f8fa"><div class="codeBlockTitle_Ktv7">https://github.com/alibaba/DataX/blob/master/mongodbreader/doc/mongodbreader.md</div><div class="codeBlockContent_biex"><pre tabindex="0" class="prism-code language-json codeBlock_bY9V thin-scrollbar"><code class="codeBlockLines_e6Vv"><span class="token-line" style="color:#393A34"><span class="token plain">  </span><span class="token punctuation" style="color:#393A34">{</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">  </span><span class="token property" style="color:#36acaa">"job"</span><span class="token operator" style="color:#393A34">:</span><span class="token plain"> </span><span class="token punctuation" style="color:#393A34">{</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">    </span><span class="token property" style="color:#36acaa">"content"</span><span class="token operator" style="color:#393A34">:</span><span class="token plain"> </span><span class="token punctuation" style="color:#393A34">[</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">      </span><span class="token punctuation" style="color:#393A34">{</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">        </span><span class="token property" style="color:#36acaa">"reader"</span><span class="token operator" style="color:#393A34">:</span><span class="token plain"> </span><span class="token punctuation" style="color:#393A34">{</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">          </span><span class="token property" style="color:#36acaa">"name"</span><span class="token operator" style="color:#393A34">:</span><span class="token plain"> </span><span class="token string" style="color:#e3116c">"mongodbreader"</span><span class="token punctuation" style="color:#393A34">,</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">          </span><span class="token property" style="color:#36acaa">"parameter"</span><span class="token operator" style="color:#393A34">:</span><span class="token plain"> </span><span class="token punctuation" style="color:#393A34">{</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">            </span><span class="token property" style="color:#36acaa">"address"</span><span class="token operator" style="color:#393A34">:</span><span class="token plain"> </span><span class="token punctuation" style="color:#393A34">[</span><span class="token string" style="color:#e3116c">"127.0.0.1:27017"</span><span class="token punctuation" style="color:#393A34">]</span><span class="token punctuation" style="color:#393A34">,</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">            </span><span class="token property" style="color:#36acaa">"dbName"</span><span class="token operator" style="color:#393A34">:</span><span class="token plain"> </span><span class="token string" style="color:#e3116c">"tag_per_data"</span><span class="token punctuation" style="color:#393A34">,</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">            </span><span class="token property" style="color:#36acaa">"collectionName"</span><span class="token operator" style="color:#393A34">:</span><span class="token plain"> </span><span class="token string" style="color:#e3116c">"tag_data12"</span><span class="token punctuation" style="color:#393A34">,</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">            </span><span class="token property" style="color:#36acaa">"column"</span><span class="token operator" style="color:#393A34">:</span><span class="token plain"> </span><span class="token punctuation" style="color:#393A34">[</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">              </span><span class="token punctuation" style="color:#393A34">{</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">                </span><span class="token property" style="color:#36acaa">"name"</span><span class="token operator" style="color:#393A34">:</span><span class="token plain"> </span><span class="token string" style="color:#e3116c">"unique_id"</span><span class="token punctuation" style="color:#393A34">,</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">                </span><span class="token property" style="color:#36acaa">"type"</span><span class="token operator" style="color:#393A34">:</span><span class="token plain"> </span><span class="token string" style="color:#e3116c">"string"</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">              </span><span class="token punctuation" style="color:#393A34">}</span><span class="token punctuation" style="color:#393A34">,</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">              </span><span class="token punctuation" style="color:#393A34">{</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">                </span><span class="token property" style="color:#36acaa">"name"</span><span class="token operator" style="color:#393A34">:</span><span class="token plain"> </span><span class="token string" style="color:#e3116c">"sid"</span><span class="token punctuation" style="color:#393A34">,</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">                </span><span class="token property" style="color:#36acaa">"type"</span><span class="token operator" style="color:#393A34">:</span><span class="token plain"> </span><span class="token string" style="color:#e3116c">"string"</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">              </span><span class="token punctuation" style="color:#393A34">}</span><span class="token punctuation" style="color:#393A34">,</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">              </span><span class="token punctuation" style="color:#393A34">{</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">                </span><span class="token property" style="color:#36acaa">"name"</span><span class="token operator" style="color:#393A34">:</span><span class="token plain"> </span><span class="token string" style="color:#e3116c">"user_id"</span><span class="token punctuation" style="color:#393A34">,</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">                </span><span class="token property" style="color:#36acaa">"type"</span><span class="token operator" style="color:#393A34">:</span><span class="token plain"> </span><span class="token string" style="color:#e3116c">"string"</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">              </span><span class="token punctuation" style="color:#393A34">}</span><span class="token punctuation" style="color:#393A34">,</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">              </span><span class="token punctuation" style="color:#393A34">{</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">                </span><span class="token property" style="color:#36acaa">"name"</span><span class="token operator" style="color:#393A34">:</span><span class="token plain"> </span><span class="token string" style="color:#e3116c">"auction_id"</span><span class="token punctuation" style="color:#393A34">,</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">                </span><span class="token property" style="color:#36acaa">"type"</span><span class="token operator" style="color:#393A34">:</span><span class="token plain"> </span><span class="token string" style="color:#e3116c">"string"</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">              </span><span class="token punctuation" style="color:#393A34">}</span><span class="token punctuation" style="color:#393A34">,</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">              </span><span class="token punctuation" style="color:#393A34">{</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">                </span><span class="token property" style="color:#36acaa">"name"</span><span class="token operator" style="color:#393A34">:</span><span class="token plain"> </span><span class="token string" style="color:#e3116c">"content_type"</span><span class="token punctuation" style="color:#393A34">,</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">                </span><span class="token property" style="color:#36acaa">"type"</span><span class="token operator" style="color:#393A34">:</span><span class="token plain"> </span><span class="token string" style="color:#e3116c">"string"</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">              </span><span class="token punctuation" style="color:#393A34">}</span><span class="token punctuation" style="color:#393A34">,</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">              </span><span class="token punctuation" style="color:#393A34">{</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">                </span><span class="token property" style="color:#36acaa">"name"</span><span class="token operator" style="color:#393A34">:</span><span class="token plain"> </span><span class="token string" style="color:#e3116c">"pool_type"</span><span class="token punctuation" style="color:#393A34">,</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">                </span><span class="token property" style="color:#36acaa">"type"</span><span class="token operator" style="color:#393A34">:</span><span class="token plain"> </span><span class="token string" style="color:#e3116c">"string"</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">              </span><span class="token punctuation" style="color:#393A34">}</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">            </span><span class="token punctuation" style="color:#393A34">]</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">          </span><span class="token punctuation" style="color:#393A34">}</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">        </span><span class="token punctuation" style="color:#393A34">}</span><span class="token punctuation" style="color:#393A34">,</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">        </span><span class="token property" style="color:#36acaa">"writer"</span><span class="token operator" style="color:#393A34">:</span><span class="token plain"> </span><span class="token punctuation" style="color:#393A34">{</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">          </span><span class="token property" style="color:#36acaa">"name"</span><span class="token operator" style="color:#393A34">:</span><span class="token plain"> </span><span class="token string" style="color:#e3116c">"odpswriter"</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">        </span><span class="token punctuation" style="color:#393A34">}</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">      </span><span class="token punctuation" style="color:#393A34">}</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">    </span><span class="token punctuation" style="color:#393A34">]</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">  </span><span class="token punctuation" style="color:#393A34">}</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain"></span><span class="token punctuation" style="color:#393A34">}</span><br></span></code></pre><div class="buttonGroup__atx"><button type="button" aria-label="Copy code to clipboard" title="Copy" class="clean-btn"><span class="copyButtonIcons_eSgA" aria-hidden="true"><svg class="copyButtonIcon_y97N" viewBox="0 0 24 24"><path d="M19,21H8V7H19M19,5H8A2,2 0 0,0 6,7V21A2,2 0 0,0 8,23H19A2,2 0 0,0 21,21V7A2,2 0 0,0 19,5M16,1H4A2,2 0 0,0 2,3V17H4V3H16V1Z"></path></svg><svg class="copyButtonSuccessIcon_LjdS" viewBox="0 0 24 24"><path d="M21,7L9,19L3.5,13.5L4.91,12.09L9,16.17L19.59,5.59L21,7Z"></path></svg></span></button></div></div></div><p>以上配置文件中 reader <code>mongodbreader</code>需要配置对应表的列枚举信息，列中存在<code>BSON</code>类型的列，还存在拆列的问题，会更加复杂，配置过程虽然简单，但还是很容易会出错，特别在配置<code>type</code>属性时。</p><h2 class="anchor anchorWithStickyNavbar_LWe7" id="优化">优化<a class="hash-link" href="#优化" title="Direct link to heading">​</a></h2><h3 class="anchor anchorWithStickyNavbar_LWe7" id="思路">思路<a class="hash-link" href="#思路" title="Direct link to heading">​</a></h3><p>优化思路，是否可以通过MongoDB的JDBC客户端，通过反射的方式得到表的列信息列表。然后通过模版机制（如：velocity）自动生成DataX配置文件中<code>column</code>配置。</p><p>尝试通过MongoDB Client API读取表Schema元数据信息，我们可以尝试从collection中读取一条记录，然后通过解析记录获得Schema记录，如下：</p><div class="language-javascript codeBlockContainer_Ckt0 theme-code-block" style="--prism-color:#393A34;--prism-background-color:#f6f8fa"><div class="codeBlockContent_biex"><pre tabindex="0" class="prism-code language-javascript codeBlock_bY9V thin-scrollbar"><code class="codeBlockLines_e6Vv"><span class="token-line" style="color:#393A34"><span class="token plain">  </span><span class="token keyword" style="color:#00009f">var</span><span class="token plain"> schemaObj </span><span class="token operator" style="color:#393A34">=</span><span class="token plain"> db</span><span class="token punctuation" style="color:#393A34">.</span><span class="token property-access">users</span><span class="token punctuation" style="color:#393A34">.</span><span class="token method function property-access" style="color:#d73a49">findOne</span><span class="token punctuation" style="color:#393A34">(</span><span class="token punctuation" style="color:#393A34">)</span><span class="token punctuation" style="color:#393A34">;</span><br></span></code></pre><div class="buttonGroup__atx"><button type="button" aria-label="Copy code to clipboard" title="Copy" class="clean-btn"><span class="copyButtonIcons_eSgA" aria-hidden="true"><svg class="copyButtonIcon_y97N" viewBox="0 0 24 24"><path d="M19,21H8V7H19M19,5H8A2,2 0 0,0 6,7V21A2,2 0 0,0 8,23H19A2,2 0 0,0 21,21V7A2,2 0 0,0 19,5M16,1H4A2,2 0 0,0 2,3V17H4V3H16V1Z"></path></svg><svg class="copyButtonSuccessIcon_LjdS" viewBox="0 0 24 24"><path d="M21,7L9,19L3.5,13.5L4.91,12.09L9,16.17L19.59,5.59L21,7Z"></path></svg></span></button></div></div></div><p>遍历记录的所有列</p><div class="language-javascript codeBlockContainer_Ckt0 theme-code-block" style="--prism-color:#393A34;--prism-background-color:#f6f8fa"><div class="codeBlockContent_biex"><pre tabindex="0" class="prism-code language-javascript codeBlock_bY9V thin-scrollbar"><code class="codeBlockLines_e6Vv"><span class="token-line" style="color:#393A34"><span class="token plain">  </span><span class="token keyword" style="color:#00009f">void</span><span class="token plain"> </span><span class="token function" style="color:#d73a49">printSchema</span><span class="token punctuation" style="color:#393A34">(</span><span class="token parameter">obj</span><span class="token punctuation" style="color:#393A34">)</span><span class="token plain"> </span><span class="token punctuation" style="color:#393A34">{</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">        </span><span class="token keyword control-flow" style="color:#00009f">for</span><span class="token plain"> </span><span class="token punctuation" style="color:#393A34">(</span><span class="token keyword" style="color:#00009f">var</span><span class="token plain"> key </span><span class="token keyword" style="color:#00009f">in</span><span class="token plain"> obj</span><span class="token punctuation" style="color:#393A34">)</span><span class="token plain"> </span><span class="token punctuation" style="color:#393A34">{</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">            </span><span class="token function" style="color:#d73a49">print</span><span class="token punctuation" style="color:#393A34">(</span><span class="token plain">indent</span><span class="token punctuation" style="color:#393A34">,</span><span class="token plain"> key</span><span class="token punctuation" style="color:#393A34">,</span><span class="token plain"> </span><span class="token keyword" style="color:#00009f">typeof</span><span class="token plain"> obj</span><span class="token punctuation" style="color:#393A34">[</span><span class="token plain">key</span><span class="token punctuation" style="color:#393A34">]</span><span class="token punctuation" style="color:#393A34">)</span><span class="token plain"> </span><span class="token punctuation" style="color:#393A34">;</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">        </span><span class="token punctuation" style="color:#393A34">}</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">  </span><span class="token punctuation" style="color:#393A34">}</span><span class="token punctuation" style="color:#393A34">;</span><br></span></code></pre><div class="buttonGroup__atx"><button type="button" aria-label="Copy code to clipboard" title="Copy" class="clean-btn"><span class="copyButtonIcons_eSgA" aria-hidden="true"><svg class="copyButtonIcon_y97N" viewBox="0 0 24 24"><path d="M19,21H8V7H19M19,5H8A2,2 0 0,0 6,7V21A2,2 0 0,0 8,23H19A2,2 0 0,0 21,21V7A2,2 0 0,0 19,5M16,1H4A2,2 0 0,0 2,3V17H4V3H16V1Z"></path></svg><svg class="copyButtonSuccessIcon_LjdS" viewBox="0 0 24 24"><path d="M21,7L9,19L3.5,13.5L4.91,12.09L9,16.17L19.59,5.59L21,7Z"></path></svg></span></button></div></div></div><p>可以将Schema打印出，如下：<br><br>
<img loading="lazy" src="/assets/images/1_9aFUkXYSRE_6ezUxH7pw3Q-f28addc41b7cd5957f034c9babae066c.webp" width="206" height="279" class="img_ev3q"></p><p>这非常酷，而用户自定义Collection往往存在子属性，希望将这些子属性进行拆接打平，可以导入到下游目标端中。</p><p>我们可以优化以上代码：</p><div class="language-javascript codeBlockContainer_Ckt0 theme-code-block" style="--prism-color:#393A34;--prism-background-color:#f6f8fa"><div class="codeBlockContent_biex"><pre tabindex="0" class="prism-code language-javascript codeBlock_bY9V thin-scrollbar"><code class="codeBlockLines_e6Vv"><span class="token-line" style="color:#393A34"><span class="token keyword" style="color:#00009f">function</span><span class="token plain"> </span><span class="token function" style="color:#d73a49">printSchema</span><span class="token punctuation" style="color:#393A34">(</span><span class="token parameter">obj</span><span class="token parameter punctuation" style="color:#393A34">,</span><span class="token parameter"> indent</span><span class="token punctuation" style="color:#393A34">)</span><span class="token plain"> </span><span class="token punctuation" style="color:#393A34">{</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">        </span><span class="token keyword control-flow" style="color:#00009f">for</span><span class="token plain"> </span><span class="token punctuation" style="color:#393A34">(</span><span class="token keyword" style="color:#00009f">var</span><span class="token plain"> key </span><span class="token keyword" style="color:#00009f">in</span><span class="token plain"> obj</span><span class="token punctuation" style="color:#393A34">)</span><span class="token plain"> </span><span class="token punctuation" style="color:#393A34">{</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">            </span><span class="token function" style="color:#d73a49">print</span><span class="token punctuation" style="color:#393A34">(</span><span class="token plain">indent</span><span class="token punctuation" style="color:#393A34">,</span><span class="token plain"> key</span><span class="token punctuation" style="color:#393A34">,</span><span class="token plain"> </span><span class="token keyword" style="color:#00009f">typeof</span><span class="token plain"> obj</span><span class="token punctuation" style="color:#393A34">[</span><span class="token plain">key</span><span class="token punctuation" style="color:#393A34">]</span><span class="token punctuation" style="color:#393A34">)</span><span class="token plain"> </span><span class="token punctuation" style="color:#393A34">;</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">            </span><span class="token keyword control-flow" style="color:#00009f">if</span><span class="token plain"> </span><span class="token punctuation" style="color:#393A34">(</span><span class="token keyword" style="color:#00009f">typeof</span><span class="token plain"> obj</span><span class="token punctuation" style="color:#393A34">[</span><span class="token plain">key</span><span class="token punctuation" style="color:#393A34">]</span><span class="token plain"> </span><span class="token operator" style="color:#393A34">==</span><span class="token plain"> </span><span class="token string" style="color:#e3116c">"object"</span><span class="token punctuation" style="color:#393A34">)</span><span class="token plain"> </span><span class="token punctuation" style="color:#393A34">{</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">                </span><span class="token function" style="color:#d73a49">printSchema</span><span class="token punctuation" style="color:#393A34">(</span><span class="token plain">obj</span><span class="token punctuation" style="color:#393A34">[</span><span class="token plain">key</span><span class="token punctuation" style="color:#393A34">]</span><span class="token punctuation" style="color:#393A34">,</span><span class="token plain"> indent </span><span class="token operator" style="color:#393A34">+</span><span class="token plain"> </span><span class="token string" style="color:#e3116c">"\t"</span><span class="token punctuation" style="color:#393A34">)</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">            </span><span class="token punctuation" style="color:#393A34">}</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">        </span><span class="token punctuation" style="color:#393A34">}</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">    </span><span class="token punctuation" style="color:#393A34">}</span><span class="token punctuation" style="color:#393A34">;</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain"></span><span class="token function" style="color:#d73a49">printSchema</span><span class="token punctuation" style="color:#393A34">(</span><span class="token plain">schemaObj</span><span class="token punctuation" style="color:#393A34">,</span><span class="token string" style="color:#e3116c">""</span><span class="token punctuation" style="color:#393A34">)</span><span class="token punctuation" style="color:#393A34">;</span><br></span></code></pre><div class="buttonGroup__atx"><button type="button" aria-label="Copy code to clipboard" title="Copy" class="clean-btn"><span class="copyButtonIcons_eSgA" aria-hidden="true"><svg class="copyButtonIcon_y97N" viewBox="0 0 24 24"><path d="M19,21H8V7H19M19,5H8A2,2 0 0,0 6,7V21A2,2 0 0,0 8,23H19A2,2 0 0,0 21,21V7A2,2 0 0,0 19,5M16,1H4A2,2 0 0,0 2,3V17H4V3H16V1Z"></path></svg><svg class="copyButtonSuccessIcon_LjdS" viewBox="0 0 24 24"><path d="M21,7L9,19L3.5,13.5L4.91,12.09L9,16.17L19.59,5.59L21,7Z"></path></svg></span></button></div></div></div><p>重新执行以上代码，将会打印：<br><br>
<img loading="lazy" alt="1_nWI-bWoJHWE1WU2Mgk8z-A.webp" src="/assets/images/1_nWI-bWoJHWE1WU2Mgk8z-A-5ae27380af9d81be2b4d3cd775d9c71c.webp" width="209" height="380" class="img_ev3q"></p><ul><li>参考： <a href="https://medium.com/@ahsan.ayaz/how-to-find-schema-of-a-collection-in-mongodb-d9a91839d992" target="_blank" rel="noopener noreferrer">https://medium.com/@ahsan.ayaz/how-to-find-schema-of-a-collection-in-mongodb-d9a91839d992</a></li></ul><h3 class="anchor anchorWithStickyNavbar_LWe7" id="在tis中具体实现">在TIS中具体实现<a class="hash-link" href="#在tis中具体实现" title="Direct link to heading">​</a></h3><p>TIS中读取MongoDB表Schema实现方式沿袭以上思路，另外添加额外的工序：</p><ol><li>在控制台中设置尝试读取的记录数，由于MongoDB Collection中每条记录的列数量和类型不一定相同的，可以尝试读取多条Collection中记录，将每条记录Schema进行Merge最终获得Schema。</li><li>读取每条记录类型为<code>BsonType.DOCUMENT</code>的列类型，将内部子列与父列通过'.'号就行连接，打平成为新的列,例如："user.name","user.age"</li><li>将获得到的Schema在前台展示，用户可以通过在表单中对Schema结构进行微调，以确认最终的Schema结构。</li></ol><p>以下为 <a href="https://github.com/qlangtech/plugins/blob/b728488cc198661e6657cc174b4326495f9d5e6d/tis-datax/tis-datax-mongodb-plugin/src/main/java/com/qlangtech/tis/plugin/datax/mongo/MongoColumnMetaData.java#L69" target="_blank" rel="noopener noreferrer">MongoColumnMetaData.java</a> GitHub中路径代码中的片段：</p><div class="language-java codeBlockContainer_Ckt0 theme-code-block" style="--prism-color:#393A34;--prism-background-color:#f6f8fa"><div class="codeBlockTitle_Ktv7">/tis-datax-mongodb-plugin/src/main/java/com/qlangtech/tis/plugin/datax/mongo/MongoColumnMetaData.java#L69</div><div class="codeBlockContent_biex"><pre tabindex="0" class="prism-code language-java codeBlock_bY9V thin-scrollbar"><code class="codeBlockLines_e6Vv"><span class="token-line" style="color:#393A34"><span class="token plain">    </span><span class="token keyword" style="color:#00009f">public</span><span class="token plain"> </span><span class="token keyword" style="color:#00009f">static</span><span class="token plain"> </span><span class="token keyword" style="color:#00009f">void</span><span class="token plain"> </span><span class="token function" style="color:#d73a49">parseMongoDocTypes</span><span class="token punctuation" style="color:#393A34">(</span><span class="token keyword" style="color:#00009f">boolean</span><span class="token plain"> parseChildDoc</span><span class="token punctuation" style="color:#393A34">,</span><span class="token plain"> </span><span class="token class-name">List</span><span class="token generics punctuation" style="color:#393A34">&lt;</span><span class="token generics class-name">String</span><span class="token generics punctuation" style="color:#393A34">&gt;</span><span class="token plain"> parentKeys </span><span class="token comment" style="color:#999988;font-style:italic">//</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">            </span><span class="token punctuation" style="color:#393A34">,</span><span class="token plain"> </span><span class="token class-name">Map</span><span class="token generics punctuation" style="color:#393A34">&lt;</span><span class="token generics class-name">String</span><span class="token generics punctuation" style="color:#393A34">,</span><span class="token generics"> </span><span class="token generics class-name">MongoColumnMetaData</span><span class="token generics punctuation" style="color:#393A34">&gt;</span><span class="token plain"> colsSchema</span><span class="token punctuation" style="color:#393A34">,</span><span class="token plain"> </span><span class="token class-name">BsonDocument</span><span class="token plain"> bdoc</span><span class="token punctuation" style="color:#393A34">)</span><span class="token plain"> </span><span class="token punctuation" style="color:#393A34">{</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">        </span><span class="token keyword" style="color:#00009f">int</span><span class="token plain"> index </span><span class="token operator" style="color:#393A34">=</span><span class="token plain"> </span><span class="token number" style="color:#36acaa">0</span><span class="token punctuation" style="color:#393A34">;</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">        </span><span class="token class-name">BsonValue</span><span class="token plain"> val</span><span class="token punctuation" style="color:#393A34">;</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">        </span><span class="token class-name">String</span><span class="token plain"> key</span><span class="token punctuation" style="color:#393A34">;</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">        </span><span class="token class-name">MongoColumnMetaData</span><span class="token plain"> colMeta</span><span class="token punctuation" style="color:#393A34">;</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">        </span><span class="token class-name">List</span><span class="token generics punctuation" style="color:#393A34">&lt;</span><span class="token generics class-name">String</span><span class="token generics punctuation" style="color:#393A34">&gt;</span><span class="token plain"> keys </span><span class="token operator" style="color:#393A34">=</span><span class="token plain"> </span><span class="token keyword" style="color:#00009f">null</span><span class="token punctuation" style="color:#393A34">;</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain" style="display:inline-block"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">        </span><span class="token keyword" style="color:#00009f">for</span><span class="token plain"> </span><span class="token punctuation" style="color:#393A34">(</span><span class="token class-name">Map</span><span class="token class-name punctuation" style="color:#393A34">.</span><span class="token class-name">Entry</span><span class="token generics punctuation" style="color:#393A34">&lt;</span><span class="token generics class-name">String</span><span class="token generics punctuation" style="color:#393A34">,</span><span class="token generics"> </span><span class="token generics class-name">BsonValue</span><span class="token generics punctuation" style="color:#393A34">&gt;</span><span class="token plain"> entry </span><span class="token operator" style="color:#393A34">:</span><span class="token plain"> bdoc</span><span class="token punctuation" style="color:#393A34">.</span><span class="token function" style="color:#d73a49">entrySet</span><span class="token punctuation" style="color:#393A34">(</span><span class="token punctuation" style="color:#393A34">)</span><span class="token punctuation" style="color:#393A34">)</span><span class="token plain"> </span><span class="token punctuation" style="color:#393A34">{</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">            val </span><span class="token operator" style="color:#393A34">=</span><span class="token plain"> entry</span><span class="token punctuation" style="color:#393A34">.</span><span class="token function" style="color:#d73a49">getValue</span><span class="token punctuation" style="color:#393A34">(</span><span class="token punctuation" style="color:#393A34">)</span><span class="token punctuation" style="color:#393A34">;</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">            keys </span><span class="token operator" style="color:#393A34">=</span><span class="token plain"> </span><span class="token class-name">ListUtils</span><span class="token punctuation" style="color:#393A34">.</span><span class="token function" style="color:#d73a49">union</span><span class="token punctuation" style="color:#393A34">(</span><span class="token plain">parentKeys</span><span class="token punctuation" style="color:#393A34">,</span><span class="token plain"> </span><span class="token class-name">Collections</span><span class="token punctuation" style="color:#393A34">.</span><span class="token function" style="color:#d73a49">singletonList</span><span class="token punctuation" style="color:#393A34">(</span><span class="token plain">entry</span><span class="token punctuation" style="color:#393A34">.</span><span class="token function" style="color:#d73a49">getKey</span><span class="token punctuation" style="color:#393A34">(</span><span class="token punctuation" style="color:#393A34">)</span><span class="token punctuation" style="color:#393A34">)</span><span class="token punctuation" style="color:#393A34">)</span><span class="token punctuation" style="color:#393A34">;</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">            key </span><span class="token operator" style="color:#393A34">=</span><span class="token plain"> </span><span class="token class-name">String</span><span class="token punctuation" style="color:#393A34">.</span><span class="token function" style="color:#d73a49">join</span><span class="token punctuation" style="color:#393A34">(</span><span class="token class-name">MongoCMeta</span><span class="token punctuation" style="color:#393A34">.</span><span class="token constant" style="color:#36acaa">KEY_MONOG_NEST_PROP_SEPERATOR</span><span class="token punctuation" style="color:#393A34">,</span><span class="token plain"> keys</span><span class="token punctuation" style="color:#393A34">)</span><span class="token punctuation" style="color:#393A34">;</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">            colMeta </span><span class="token operator" style="color:#393A34">=</span><span class="token plain"> colsSchema</span><span class="token punctuation" style="color:#393A34">.</span><span class="token function" style="color:#d73a49">get</span><span class="token punctuation" style="color:#393A34">(</span><span class="token plain">key</span><span class="token punctuation" style="color:#393A34">)</span><span class="token punctuation" style="color:#393A34">;</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">            </span><span class="token keyword" style="color:#00009f">if</span><span class="token plain"> </span><span class="token punctuation" style="color:#393A34">(</span><span class="token plain">colMeta </span><span class="token operator" style="color:#393A34">==</span><span class="token plain"> </span><span class="token keyword" style="color:#00009f">null</span><span class="token punctuation" style="color:#393A34">)</span><span class="token plain"> </span><span class="token punctuation" style="color:#393A34">{</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">                colMeta </span><span class="token operator" style="color:#393A34">=</span><span class="token plain"> </span><span class="token keyword" style="color:#00009f">new</span><span class="token plain"> </span><span class="token class-name">MongoColumnMetaData</span><span class="token punctuation" style="color:#393A34">(</span><span class="token plain">index</span><span class="token operator" style="color:#393A34">++</span><span class="token punctuation" style="color:#393A34">,</span><span class="token plain"> key</span><span class="token punctuation" style="color:#393A34">,</span><span class="token plain"> val</span><span class="token punctuation" style="color:#393A34">.</span><span class="token function" style="color:#d73a49">getBsonType</span><span class="token punctuation" style="color:#393A34">(</span><span class="token punctuation" style="color:#393A34">)</span><span class="token punctuation" style="color:#393A34">,</span><span class="token plain"> </span><span class="token number" style="color:#36acaa">0</span><span class="token punctuation" style="color:#393A34">,</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">                        </span><span class="token punctuation" style="color:#393A34">(</span><span class="token plain">val</span><span class="token punctuation" style="color:#393A34">.</span><span class="token function" style="color:#d73a49">getBsonType</span><span class="token punctuation" style="color:#393A34">(</span><span class="token punctuation" style="color:#393A34">)</span><span class="token plain"> </span><span class="token operator" style="color:#393A34">==</span><span class="token plain"> </span><span class="token class-name">BsonType</span><span class="token punctuation" style="color:#393A34">.</span><span class="token constant" style="color:#36acaa">OBJECT_ID</span><span class="token punctuation" style="color:#393A34">)</span><span class="token punctuation" style="color:#393A34">)</span><span class="token punctuation" style="color:#393A34">;</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">                colsSchema</span><span class="token punctuation" style="color:#393A34">.</span><span class="token function" style="color:#d73a49">put</span><span class="token punctuation" style="color:#393A34">(</span><span class="token plain">key</span><span class="token punctuation" style="color:#393A34">,</span><span class="token plain"> colMeta</span><span class="token punctuation" style="color:#393A34">)</span><span class="token punctuation" style="color:#393A34">;</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">            </span><span class="token punctuation" style="color:#393A34">}</span><span class="token plain"> </span><span class="token keyword" style="color:#00009f">else</span><span class="token plain"> </span><span class="token punctuation" style="color:#393A34">{</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">                </span><span class="token keyword" style="color:#00009f">if</span><span class="token plain"> </span><span class="token punctuation" style="color:#393A34">(</span><span class="token plain">colMeta</span><span class="token punctuation" style="color:#393A34">.</span><span class="token function" style="color:#d73a49">getMongoFieldType</span><span class="token punctuation" style="color:#393A34">(</span><span class="token punctuation" style="color:#393A34">)</span><span class="token plain"> </span><span class="token operator" style="color:#393A34">!=</span><span class="token plain"> </span><span class="token class-name">BsonType</span><span class="token punctuation" style="color:#393A34">.</span><span class="token constant" style="color:#36acaa">STRING</span><span class="token plain"> </span><span class="token comment" style="color:#999988;font-style:italic">//</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">                        </span><span class="token operator" style="color:#393A34">&amp;&amp;</span><span class="token plain"> </span><span class="token operator" style="color:#393A34">!</span><span class="token plain">val</span><span class="token punctuation" style="color:#393A34">.</span><span class="token function" style="color:#d73a49">isNull</span><span class="token punctuation" style="color:#393A34">(</span><span class="token punctuation" style="color:#393A34">)</span><span class="token plain"> </span><span class="token operator" style="color:#393A34">&amp;&amp;</span><span class="token plain"> colMeta</span><span class="token punctuation" style="color:#393A34">.</span><span class="token function" style="color:#d73a49">getMongoFieldType</span><span class="token punctuation" style="color:#393A34">(</span><span class="token punctuation" style="color:#393A34">)</span><span class="token plain"> </span><span class="token operator" style="color:#393A34">!=</span><span class="token plain"> val</span><span class="token punctuation" style="color:#393A34">.</span><span class="token function" style="color:#d73a49">getBsonType</span><span class="token punctuation" style="color:#393A34">(</span><span class="token punctuation" style="color:#393A34">)</span><span class="token punctuation" style="color:#393A34">)</span><span class="token plain"> </span><span class="token punctuation" style="color:#393A34">{</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">                    </span><span class="token comment" style="color:#999988;font-style:italic">//TODO： 前后两次类型不同</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">                    </span><span class="token comment" style="color:#999988;font-style:italic">// 则直接将类型改成String类型</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">                    colMeta </span><span class="token operator" style="color:#393A34">=</span><span class="token plain"> </span><span class="token keyword" style="color:#00009f">new</span><span class="token plain"> </span><span class="token class-name">MongoColumnMetaData</span><span class="token punctuation" style="color:#393A34">(</span><span class="token plain">index</span><span class="token operator" style="color:#393A34">++</span><span class="token punctuation" style="color:#393A34">,</span><span class="token plain"> key</span><span class="token punctuation" style="color:#393A34">,</span><span class="token plain"> </span><span class="token class-name">BsonType</span><span class="token punctuation" style="color:#393A34">.</span><span class="token constant" style="color:#36acaa">STRING</span><span class="token punctuation" style="color:#393A34">)</span><span class="token punctuation" style="color:#393A34">;</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">                    colsSchema</span><span class="token punctuation" style="color:#393A34">.</span><span class="token function" style="color:#d73a49">put</span><span class="token punctuation" style="color:#393A34">(</span><span class="token plain">key</span><span class="token punctuation" style="color:#393A34">,</span><span class="token plain"> colMeta</span><span class="token punctuation" style="color:#393A34">)</span><span class="token punctuation" style="color:#393A34">;</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">                </span><span class="token punctuation" style="color:#393A34">}</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">            </span><span class="token punctuation" style="color:#393A34">}</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">            </span><span class="token keyword" style="color:#00009f">if</span><span class="token plain"> </span><span class="token punctuation" style="color:#393A34">(</span><span class="token operator" style="color:#393A34">!</span><span class="token plain">val</span><span class="token punctuation" style="color:#393A34">.</span><span class="token function" style="color:#d73a49">isNull</span><span class="token punctuation" style="color:#393A34">(</span><span class="token punctuation" style="color:#393A34">)</span><span class="token punctuation" style="color:#393A34">)</span><span class="token plain"> </span><span class="token punctuation" style="color:#393A34">{</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">                </span><span class="token keyword" style="color:#00009f">if</span><span class="token plain"> </span><span class="token punctuation" style="color:#393A34">(</span><span class="token plain">colMeta</span><span class="token punctuation" style="color:#393A34">.</span><span class="token function" style="color:#d73a49">getMongoFieldType</span><span class="token punctuation" style="color:#393A34">(</span><span class="token punctuation" style="color:#393A34">)</span><span class="token plain"> </span><span class="token operator" style="color:#393A34">==</span><span class="token plain"> </span><span class="token class-name">BsonType</span><span class="token punctuation" style="color:#393A34">.</span><span class="token constant" style="color:#36acaa">DOCUMENT</span><span class="token plain"> </span><span class="token operator" style="color:#393A34">&amp;&amp;</span><span class="token plain"> val</span><span class="token punctuation" style="color:#393A34">.</span><span class="token function" style="color:#d73a49">isDocument</span><span class="token punctuation" style="color:#393A34">(</span><span class="token punctuation" style="color:#393A34">)</span><span class="token punctuation" style="color:#393A34">)</span><span class="token plain"> </span><span class="token punctuation" style="color:#393A34">{</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">                    </span><span class="token function" style="color:#d73a49">parseMongoDocTypes</span><span class="token punctuation" style="color:#393A34">(</span><span class="token boolean" style="color:#36acaa">true</span><span class="token punctuation" style="color:#393A34">,</span><span class="token plain"> keys</span><span class="token punctuation" style="color:#393A34">,</span><span class="token plain"> parseChildDoc </span><span class="token operator" style="color:#393A34">?</span><span class="token plain"> colsSchema </span><span class="token operator" style="color:#393A34">:</span><span class="token plain"> colMeta</span><span class="token punctuation" style="color:#393A34">.</span><span class="token plain">docTypeFieldEnum</span><span class="token punctuation" style="color:#393A34">,</span><span class="token plain"> val</span><span class="token punctuation" style="color:#393A34">.</span><span class="token function" style="color:#d73a49">asDocument</span><span class="token punctuation" style="color:#393A34">(</span><span class="token punctuation" style="color:#393A34">)</span><span class="token punctuation" style="color:#393A34">)</span><span class="token punctuation" style="color:#393A34">;</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">                </span><span class="token punctuation" style="color:#393A34">}</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">                </span><span class="token keyword" style="color:#00009f">if</span><span class="token plain"> </span><span class="token punctuation" style="color:#393A34">(</span><span class="token plain">colMeta</span><span class="token punctuation" style="color:#393A34">.</span><span class="token function" style="color:#d73a49">getMongoFieldType</span><span class="token punctuation" style="color:#393A34">(</span><span class="token punctuation" style="color:#393A34">)</span><span class="token plain"> </span><span class="token operator" style="color:#393A34">==</span><span class="token plain"> </span><span class="token class-name">BsonType</span><span class="token punctuation" style="color:#393A34">.</span><span class="token constant" style="color:#36acaa">STRING</span><span class="token punctuation" style="color:#393A34">)</span><span class="token plain"> </span><span class="token punctuation" style="color:#393A34">{</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">                    colMeta</span><span class="token punctuation" style="color:#393A34">.</span><span class="token function" style="color:#d73a49">setMaxStrLength</span><span class="token punctuation" style="color:#393A34">(</span><span class="token plain">val</span><span class="token punctuation" style="color:#393A34">.</span><span class="token function" style="color:#d73a49">asString</span><span class="token punctuation" style="color:#393A34">(</span><span class="token punctuation" style="color:#393A34">)</span><span class="token punctuation" style="color:#393A34">.</span><span class="token function" style="color:#d73a49">getValue</span><span class="token punctuation" style="color:#393A34">(</span><span class="token punctuation" style="color:#393A34">)</span><span class="token punctuation" style="color:#393A34">.</span><span class="token function" style="color:#d73a49">length</span><span class="token punctuation" style="color:#393A34">(</span><span class="token punctuation" style="color:#393A34">)</span><span class="token punctuation" style="color:#393A34">)</span><span class="token punctuation" style="color:#393A34">;</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">                </span><span class="token punctuation" style="color:#393A34">}</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">                colMeta</span><span class="token punctuation" style="color:#393A34">.</span><span class="token function" style="color:#d73a49">incrContainValCount</span><span class="token punctuation" style="color:#393A34">(</span><span class="token punctuation" style="color:#393A34">)</span><span class="token punctuation" style="color:#393A34">;</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">            </span><span class="token punctuation" style="color:#393A34">}</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain" style="display:inline-block"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">        </span><span class="token punctuation" style="color:#393A34">}</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">    </span><span class="token punctuation" style="color:#393A34">}</span><br></span></code></pre><div class="buttonGroup__atx"><button type="button" aria-label="Copy code to clipboard" title="Copy" class="clean-btn"><span class="copyButtonIcons_eSgA" aria-hidden="true"><svg class="copyButtonIcon_y97N" viewBox="0 0 24 24"><path d="M19,21H8V7H19M19,5H8A2,2 0 0,0 6,7V21A2,2 0 0,0 8,23H19A2,2 0 0,0 21,21V7A2,2 0 0,0 19,5M16,1H4A2,2 0 0,0 2,3V17H4V3H16V1Z"></path></svg><svg class="copyButtonSuccessIcon_LjdS" viewBox="0 0 24 24"><path d="M21,7L9,19L3.5,13.5L4.91,12.09L9,16.17L19.59,5.59L21,7Z"></path></svg></span></button></div></div></div><p>MongoDB Reader 页面设置<code>预读记录数</code>，尝试读取Collection记录条数以分析出Collection的Schema，为确保最终Schema准确，可以适当将<code>预读记录数</code>设置得大一些。</p><figure><a data-fancybox="true" href="/assets/images/set-count-of-ready-read-c4cdc482a417ac2785400ebb70000632.png"><img src="/assets/images/set-count-of-ready-read-c4cdc482a417ac2785400ebb70000632.png"></a></figure><p>通过TIS解析得到的Schema会在以下页面中展示结果，用户可进行微调，确定是否需要该列，变更字段类型等。</p><figure><a data-fancybox="true" href="/assets/images/confirm-collection-schema-487e3cdc18d210f34c5d82d983153068.png"><img src="/assets/images/confirm-collection-schema-487e3cdc18d210f34c5d82d983153068.png"></a></figure>通过以上优化，最终完成数据同步通道定义，可以有效避免用户手动输入MongoDB表 Schema，达到了最大限度地降低了出错概率，且提高了工作效率<h2 class="anchor anchorWithStickyNavbar_LWe7" id="总结">总结<a class="hash-link" href="#总结" title="Direct link to heading">​</a></h2><p>数据集成领域，有大部分的端类型是和MongoDB类似属于 Schemaless的，例如Kafka，Redis，基于文件的Hdfs，FTP等等。不像MySQL这样的具有明确预定义Schema的数据源可以通过读取MetaData的方式得到Schema，从而进行自动化操作。</p><p>TIS的初衷是构建一款高度傻瓜化的DataOps数据集成软件，面向一线非技术人员，他们精通业务，在具体操作过程中不需要了解具体MongoDB中的字段类型，有哪些字段。
整个操作流程，只需要轻点鼠标，TIS会帮助用户自动生成所需配置。</p><p>借鉴这种操作思路，可以扩展到其他Schemaless的数据端读取流程上，例如Kafka，Redis，基于文件的Hdfs，FTP，将会极大地提高执行数据集成的效率。</p>]]></content>
        <author>
            <name>百岁</name>
            <uri>https://github.com/baisui1981</uri>
        </author>
        <category label="mongodb" term="mongodb"/>
    </entry>
    <entry>
        <title type="html"><![CDATA[内嵌插件复用]]></title>
        <id>nest-plugin-reuse</id>
        <link href="https://tis.pub/blog/nest-plugin-reuse"/>
        <updated>2024-04-22T00:00:00.000Z</updated>
        <summary type="html"><![CDATA[前言]]></summary>
        <content type="html"><![CDATA[<h2 class="anchor anchorWithStickyNavbar_LWe7" id="前言">前言<a class="hash-link" href="#前言" title="Direct link to heading">​</a></h2><p>在TIS 4.0.0 版本主要的功能是将原先但节点运行的组件扩展到分布式云环境中，以下类图中有三个组件需要依赖到<code>ServerPortExport</code>组件，</p><ol><li>Kubernete Powerjob Server</li><li>Kubernete Flink Session</li><li>Kubernete Flink Application</li></ol><p>ServerPortExport 组件负责在K8S组件（ReplicaSet）发布过程中将目标端口以不同的方式发布（Ingress，LoadBalance，NodePort）</p><iframe width="100%" height="500px" name="embed_dom" frameborder="0" src="https://www.processon.com/embed/6626299e51b84f0dfc92128c?cid=6626299e51b84f0dfc92128f"></iframe><p><a href="https://www.processon.com/diagraming/6626299e51b84f0dfc92128c" target="_blank" rel="noopener noreferrer">编辑</a></p><h2 class="anchor anchorWithStickyNavbar_LWe7" id="遇到问题">遇到问题<a class="hash-link" href="#遇到问题" title="Direct link to heading">​</a></h2><p><code>ServerPortExport</code> 组件聚合到不同的组件中，在具体运行过程中需要根据聚合类的不同有不同的初始值，</p><p>例如，聚合在K8SDataXPowerJobServer类中初始值为<code>7700</code>，而当聚合在<code>BasicFlinkK8SClusterCfg</code>中的初始值为<code>8081</code>，当然，直观来说，最简单的办法是根据聚合到不同的类，创建不同的<code>ServerPortExport</code>的子类从而来设置不同的初始值，
但这会创建出大量的冗余代码，所以，并不可取。</p><h2 class="anchor anchorWithStickyNavbar_LWe7" id="解决办法">解决办法<a class="hash-link" href="#解决办法" title="Direct link to heading">​</a></h2><p>在运行期，根据所在聚合类的Descriptor来动态设置 <code>ServerPortExport.serverPort</code> 属性的值</p><iframe width="100%" height="500px" frameborder="0" src="https://www.processon.com/embed/662637256b32731aec0d8fe0?cid=662637256b32731aec0d8fe3"></iframe><p><a href="https://www.processon.com/diagraming/662637256b32731aec0d8fe0" target="_blank" rel="noopener noreferrer">编辑</a></p><p>具体需要做以下功能：</p><ol><li>创建 DefaultExportPortProvider接口，get方法返回对应的端口默认值</li><li><code>BasicFlinkK8SClusterCfg</code>和<code>K8SDataXPowerJobServer</code>对应的 Descriptor分别实现以上接口</li><li>在运行期将Descriptor序列化成Json步骤中，需要将Descriptor实例与当前运行的线程绑定，这部分功能在Json序列化过程中执行，为此需要添加新类<code>DescriptorsJSONResult</code></li><li>为类<code>DescriptorsJSONResult</code>注册到Json序列化注册器中<div class="language-java codeBlockContainer_Ckt0 theme-code-block" style="--prism-color:#393A34;--prism-background-color:#f6f8fa"><div class="codeBlockTitle_Ktv7">JsonUtil.java</div><div class="codeBlockContent_biex"><pre tabindex="0" class="prism-code language-java codeBlock_bY9V thin-scrollbar"><code class="codeBlockLines_e6Vv"><span class="token-line" style="color:#393A34"><span class="token plain">   </span><span class="token class-name">ObjectWriter</span><span class="token plain"> descSerializer </span><span class="token operator" style="color:#393A34">=</span><span class="token plain"> </span><span class="token keyword" style="color:#00009f">new</span><span class="token plain"> </span><span class="token class-name">ObjectWriter</span><span class="token punctuation" style="color:#393A34">(</span><span class="token punctuation" style="color:#393A34">)</span><span class="token plain"> </span><span class="token punctuation" style="color:#393A34">{</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">        </span><span class="token annotation punctuation" style="color:#393A34">@Override</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">        </span><span class="token keyword" style="color:#00009f">public</span><span class="token plain"> </span><span class="token keyword" style="color:#00009f">void</span><span class="token plain"> </span><span class="token function" style="color:#d73a49">write</span><span class="token punctuation" style="color:#393A34">(</span><span class="token class-name">JSONWriter</span><span class="token plain"> jsonWriter</span><span class="token punctuation" style="color:#393A34">,</span><span class="token plain"> </span><span class="token class-name">Object</span><span class="token plain"> object</span><span class="token punctuation" style="color:#393A34">,</span><span class="token plain"> </span><span class="token class-name">Object</span><span class="token plain"> fieldName</span><span class="token punctuation" style="color:#393A34">,</span><span class="token plain"> </span><span class="token class-name">Type</span><span class="token plain"> fieldType</span><span class="token punctuation" style="color:#393A34">,</span><span class="token plain"> </span><span class="token keyword" style="color:#00009f">long</span><span class="token plain"> features</span><span class="token punctuation" style="color:#393A34">)</span><span class="token plain"> </span><span class="token punctuation" style="color:#393A34">{</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">            </span><span class="token class-name">DescriptorsJSONResult</span><span class="token plain"> value </span><span class="token operator" style="color:#393A34">=</span><span class="token plain"> </span><span class="token punctuation" style="color:#393A34">(</span><span class="token class-name">DescriptorsJSONResult</span><span class="token punctuation" style="color:#393A34">)</span><span class="token plain"> object</span><span class="token punctuation" style="color:#393A34">;</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">            </span><span class="token class-name">Objects</span><span class="token punctuation" style="color:#393A34">.</span><span class="token function" style="color:#d73a49">requireNonNull</span><span class="token punctuation" style="color:#393A34">(</span><span class="token plain">value</span><span class="token punctuation" style="color:#393A34">,</span><span class="token plain"> </span><span class="token string" style="color:#e3116c">"callable of "</span><span class="token plain"> </span><span class="token operator" style="color:#393A34">+</span><span class="token plain"> fieldName </span><span class="token operator" style="color:#393A34">+</span><span class="token plain"> </span><span class="token string" style="color:#e3116c">" can not be null"</span><span class="token punctuation" style="color:#393A34">)</span><span class="token punctuation" style="color:#393A34">;</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">            jsonWriter</span><span class="token punctuation" style="color:#393A34">.</span><span class="token function" style="color:#d73a49">writeRaw</span><span class="token punctuation" style="color:#393A34">(</span><span class="token plain">value</span><span class="token punctuation" style="color:#393A34">.</span><span class="token function" style="color:#d73a49">toJSONString</span><span class="token punctuation" style="color:#393A34">(</span><span class="token punctuation" style="color:#393A34">)</span><span class="token punctuation" style="color:#393A34">)</span><span class="token punctuation" style="color:#393A34">;</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">        </span><span class="token punctuation" style="color:#393A34">}</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">    </span><span class="token punctuation" style="color:#393A34">}</span><span class="token punctuation" style="color:#393A34">;</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">    com</span><span class="token punctuation" style="color:#393A34">.</span><span class="token plain">alibaba</span><span class="token punctuation" style="color:#393A34">.</span><span class="token plain">fastjson2</span><span class="token punctuation" style="color:#393A34">.</span><span class="token constant" style="color:#36acaa">JSON</span><span class="token punctuation" style="color:#393A34">.</span><span class="token function" style="color:#d73a49">register</span><span class="token punctuation" style="color:#393A34">(</span><span class="token class-name">DescriptorsJSONResult</span><span class="token punctuation" style="color:#393A34">.</span><span class="token keyword" style="color:#00009f">class</span><span class="token punctuation" style="color:#393A34">,</span><span class="token plain"> descSerializer</span><span class="token punctuation" style="color:#393A34">)</span><span class="token punctuation" style="color:#393A34">;</span><br></span></code></pre><div class="buttonGroup__atx"><button type="button" aria-label="Copy code to clipboard" title="Copy" class="clean-btn"><span class="copyButtonIcons_eSgA" aria-hidden="true"><svg class="copyButtonIcon_y97N" viewBox="0 0 24 24"><path d="M19,21H8V7H19M19,5H8A2,2 0 0,0 6,7V21A2,2 0 0,0 8,23H19A2,2 0 0,0 21,21V7A2,2 0 0,0 19,5M16,1H4A2,2 0 0,0 2,3V17H4V3H16V1Z"></path></svg><svg class="copyButtonSuccessIcon_LjdS" viewBox="0 0 24 24"><path d="M21,7L9,19L3.5,13.5L4.91,12.09L9,16.17L19.59,5.59L21,7Z"></path></svg></span></button></div></div></div></li><li>通过<code>ServerPortExport.json</code>配置描述文件，设置属性serverPort的默认值<div class="language-json codeBlockContainer_Ckt0 theme-code-block" style="--prism-color:#393A34;--prism-background-color:#f6f8fa"><div class="codeBlockTitle_Ktv7">ServerPortExport.json</div><div class="codeBlockContent_biex"><pre tabindex="0" class="prism-code language-json codeBlock_bY9V thin-scrollbar"><code class="codeBlockLines_e6Vv"><span class="token-line" style="color:#393A34"><span class="token plain">  </span><span class="token punctuation" style="color:#393A34">{</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">   </span><span class="token property" style="color:#36acaa">"serverPort"</span><span class="token operator" style="color:#393A34">:</span><span class="token plain"> </span><span class="token punctuation" style="color:#393A34">{</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">      </span><span class="token property" style="color:#36acaa">"help"</span><span class="token operator" style="color:#393A34">:</span><span class="token plain"> </span><span class="token string" style="color:#e3116c">"SpringBoot配置，HTTP端口号，默认7700，不建议更改"</span><span class="token punctuation" style="color:#393A34">,</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">      </span><span class="token property" style="color:#36acaa">"dftVal"</span><span class="token operator" style="color:#393A34">:</span><span class="token plain"> </span><span class="token string" style="color:#e3116c">"com.qlangtech.tis.plugin.datax.powerjob.ServerPortExport.dftExportPort():uncache_true"</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">    </span><span class="token punctuation" style="color:#393A34">}</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">  </span><span class="token punctuation" style="color:#393A34">}</span><br></span></code></pre><div class="buttonGroup__atx"><button type="button" aria-label="Copy code to clipboard" title="Copy" class="clean-btn"><span class="copyButtonIcons_eSgA" aria-hidden="true"><svg class="copyButtonIcon_y97N" viewBox="0 0 24 24"><path d="M19,21H8V7H19M19,5H8A2,2 0 0,0 6,7V21A2,2 0 0,0 8,23H19A2,2 0 0,0 21,21V7A2,2 0 0,0 19,5M16,1H4A2,2 0 0,0 2,3V17H4V3H16V1Z"></path></svg><svg class="copyButtonSuccessIcon_LjdS" viewBox="0 0 24 24"><path d="M21,7L9,19L3.5,13.5L4.91,12.09L9,16.17L19.59,5.59L21,7Z"></path></svg></span></button></div></div></div><div class="language-java codeBlockContainer_Ckt0 theme-code-block" style="--prism-color:#393A34;--prism-background-color:#f6f8fa"><div class="codeBlockTitle_Ktv7">ServerPortExport.java</div><div class="codeBlockContent_biex"><pre tabindex="0" class="prism-code language-java codeBlock_bY9V thin-scrollbar"><code class="codeBlockLines_e6Vv"><span class="token-line" style="color:#393A34"><span class="token plain">  </span><span class="token keyword" style="color:#00009f">public</span><span class="token plain"> </span><span class="token keyword" style="color:#00009f">static</span><span class="token plain"> </span><span class="token class-name">Integer</span><span class="token plain"> </span><span class="token function" style="color:#d73a49">dftExportPort</span><span class="token punctuation" style="color:#393A34">(</span><span class="token punctuation" style="color:#393A34">)</span><span class="token plain"> </span><span class="token punctuation" style="color:#393A34">{</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">    </span><span class="token keyword" style="color:#00009f">return</span><span class="token plain"> </span><span class="token punctuation" style="color:#393A34">(</span><span class="token punctuation" style="color:#393A34">(</span><span class="token class-name">DefaultExportPortProvider</span><span class="token punctuation" style="color:#393A34">)</span><span class="token plain"> </span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">       </span><span class="token class-name">DescriptorsJSONResult</span><span class="token punctuation" style="color:#393A34">.</span><span class="token function" style="color:#d73a49">getRootDescInstance</span><span class="token punctuation" style="color:#393A34">(</span><span class="token punctuation" style="color:#393A34">)</span><span class="token punctuation" style="color:#393A34">)</span><span class="token punctuation" style="color:#393A34">.</span><span class="token function" style="color:#d73a49">get</span><span class="token punctuation" style="color:#393A34">(</span><span class="token punctuation" style="color:#393A34">)</span><span class="token punctuation" style="color:#393A34">;</span><span class="token plain"></span><br></span><span class="token-line" style="color:#393A34"><span class="token plain">  </span><span class="token punctuation" style="color:#393A34">}</span><br></span></code></pre><div class="buttonGroup__atx"><button type="button" aria-label="Copy code to clipboard" title="Copy" class="clean-btn"><span class="copyButtonIcons_eSgA" aria-hidden="true"><svg class="copyButtonIcon_y97N" viewBox="0 0 24 24"><path d="M19,21H8V7H19M19,5H8A2,2 0 0,0 6,7V21A2,2 0 0,0 8,23H19A2,2 0 0,0 21,21V7A2,2 0 0,0 19,5M16,1H4A2,2 0 0,0 2,3V17H4V3H16V1Z"></path></svg><svg class="copyButtonSuccessIcon_LjdS" viewBox="0 0 24 24"><path d="M21,7L9,19L3.5,13.5L4.91,12.09L9,16.17L19.59,5.59L21,7Z"></path></svg></span></button></div></div></div></li></ol><h2 class="anchor anchorWithStickyNavbar_LWe7" id="总结">总结<a class="hash-link" href="#总结" title="Direct link to heading">​</a></h2><p>   通过以上步骤，就可以将<code>ServerPortExport</code>根据所在聚合类不同将属性<code>serverPort</code>初始化成不同的默认值。
以此作为一个例子，可以在TIS中相同需求可以推而广之。</p>]]></content>
        <author>
            <name>百岁</name>
            <uri>https://github.com/baisui1981</uri>
        </author>
        <category label="plugin" term="plugin"/>
    </entry>
</feed>