Package path implements utility routines for manipulating slash-separated paths. The path package should only be used for paths separated by forward slashes, such as the paths in URLs. This package does not deal with Windows paths with drive letters or backslashes; to manipulate operating system paths, use the path/filepath package.
]]><p><code>Go</code>标准库中自带<code>path包</code>使用说明：</p>
<blockquote>
<p>Package path implements utility routines for manipulating slash-separated paths.<br>The path package should only be used for paths separated by forward slashes, such as the paths in URLs. This package does not deal with Windows paths with drive letters or backslashes; to manipulate operating system paths, use the path/filepath package.<br>GoLang 中 Json Tag用法汇总http://fivezh.github.io/2022/02/01/golang-json-tag/2022-02-01T07:42:43.000Z2022-02-06T14:21:55.000ZGoLang中结构体的 JSON Tag 标识（英文名backquote或backtick，反引号 ` 符号包裹的部分内容）一直未明确看过完整规范和使用说明，存在模棱两可，系统整理如下：
// Field appears in JSON as key "myName". // 该字段出现在JSON中时，名为"myName" Field int`json:"myName"`
// Field appears in JSON as key "myName" and // the field is omitted from the object if its value is empty, // as defined above. // 该字段出现在JSON中时，名为"myName"，如为零值则输出时忽略此字段 Field int`json:"myName,omitempty"`
// Field appears in JSON as key "Field" (the default), but // the field is skipped if empty. // Note the leading comma. // 该字段出现在JSON中时，名为"Field"，如为零值则输出时忽略此字段 Field int`json:",omitempty"`
// Field is ignored by this package. // 该字段当前包输出JSON时忽略 Field int`json:"-"`
// Field appears in JSON as key "-". // 该字段出现在JSON中时，名为"-" Field int`json:"-,"`
我们正在将 Clickhouse 用于 ApiRoad.net 项目（这是一个 API 市场，开发人员出售其 API ，目前活跃开发中）的日志存储和分析，到目前为止，我们对效果感到满意。作为一名 API 开发人员， HTTP 请求/响应周期的可观察性和可分析性对于维护服务质量和快速发现 bug 非常重要，这一点对于纯 API 服务尤其如此。
SELECT a.timePeriod as t, b.count as c from ( with (select toUInt32(dateDiff('day', [START_DATE], [END_DATE])) ) as diffInTimeUnits select arrayJoin(arrayMap(x -> (toDate(addDays([START_DATE], x))), range(0, diffInTimeUnits+1))) as timePeriod ) a LEFT JOIN (select count(*) as count, toDate(toStartOfDay(started_at)) as timePeriod from logs WHERE [CONDITIONS] GROUP BY toStartOfDay(started_at)) b on a.timePeriod=b.timePeriod
在这里，我们通过 lambda 函数和循环生成一个虚拟表，然后再与按天分组的日志表进行左连接。
我认为 arrayJoin + arrayMap + range 函数方式相比 generate_series() 有更多灵活性。通过 WITH FILL 关键词可用于更简洁的语法。
MySQL 时，在仅有1亿行日志数据的表上进行如索引失效的非最优查询，都会使服务器陷入缓慢并产生内存交换，因此 MySQL 并不适合大型日志查询。但就存储而言，压缩的 InnoDB 表并没有那么糟糕。由于其基于行的特性，与 Clickhouse 相比，数据压缩方面的情况要差得多（抱歉，这次没有相关链接），但是它仍然可以在不显著降低性能的情况下设法降低成本 。因此在某些情况下，对于少量日志来说，我们依然可以使用 InnoDB 表。
4. 统计函数
在 Clickhouse 中，很容易计算 404 查询的中位数和 99 分位的耗时：
1 2 3 4 5
SELECT count(*) as cnt, quantileTiming(0.5)(duration) as duration_median, quantileTiming(0.9)(duration) as duration_90th, quantileTiming(0.99)(duration) as duration_99th FROM logs WHERE status=404
The built-in function new takes a type T, allocates storage for a variable of that type at run time, and returns a value of type *T pointing to it.The variable is initialized as described in the section on initial values.
The built-in function make takes a type T, which must be a slice, map or channel type, optionally followed by a type-specific list of expressions. It returns a value of type T (not *T). The memory is initialized as described in the section on initial values.
// nil maps var m map[t]u // nil map m2 := map[t]u{} // empty map len(m) // 0 forrange m // iterates zero times v, ok := m[i] // zero(u), false m[i] = x // panic: assignment to entry in nil map
var p *Person // nil of type *Person var s fmt.Stringer = p // Stringer (*Person, nil) fmt.Printf("%#v\n", s) // 注意，此时打印输出为(*Person)(nil)，无任何是interface的体现 fmt.Println(s == nil) // false
不要返回具体的错误类型，而应直接返回nil
下面展示返回类型为interface时的差异：
错误示例：
1 2 3 4 5 6 7 8 9
funcdo()error { // error(*doError, nil) var err *doError return err // nil of type *doError }
var c chan t // nil of type chan t // nil channel操作时 <- c // block forever，持续阻塞 c <- x // block forever，持续阻塞 close(c) // panic: close of nil channel，关闭nil channel发生panic
// 对于已关闭的channel，将发生如下现象 v, ok := <-c // zero(t), false 不会阻塞，返回零值和False c <-x // panic: send to close channel close(x) // panic: close of closed channel，备注原文中错误
funcmerge(out chan<- int, a, b <-chanint) { for a != nil || b != nil { select { case v, ok := <-a: // 此时通道关闭后，就不会再进行获取了 if !ok { a = nil fmt.Println("a is closed") continue } out <- v case v, ok := <-b: if !ok { b = nil fmt.Println("b is closed") continue } out <- v } } close(out) // 需要再不使用后进行close操作 }
nil func用法
go中函数是一等公民
函数可以作为struct结构体的字段，缺省值则为nil
1 2 3 4 5 6 7 8 9 10 11
type Foo struct { f func()error // fistypeoffunc()error } // 常见用法，传输函数为nil，增加缺省处理 funcNewServer(logger func(string, ...interface{})) { if logger == nil { logger = log.Printf } logger.Printf("initializng %s", os.Getenv("hostname")) }
var t *tree // nil of type *tree var s Summer = t // nil指针，可以是合法的interface类型的值 // 此时，对接接口类型变量s而言，其类型为*tree，值为nil，也就是说(*tree, nil)行的interface fmt.Println(t==nil, s.Sum()) // true, 0
type ints []int func(i *ints)Sum()int { s := 0 for _, v := range i{ s += v } return s }
var i ints var s Sumer = i // nil value can satisfy interface fmt.Println(s==nil, s.Sum()) // true, 0
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17
// 通过判断接口为nil，来给定缺省值 funcdoSum(s Summer)int { if s == nil { return0 } return s.Sum() }
var t *tree doSum(t) // interface的类型和值分别为：(*tree, nil)
RECORD LOCKS space id 58 page no 3 n bits 72 index `PRIMARY` of table xxxx trx id 10078 lock_mode X locks rec but not gap Record lock, heapno2PHYSICALRECORD: n_fields 3; compact format; info bits 0 0: len 4; hex 8000000a; asc ;; 1: len 6; hex 00000000274f; asc 'O;; 2: len 7; hex b60000019d0110; asc ;;
Gap Lock：范围锁定
按照范围锁定索引记录
例如，SELECT c1 FROM t WHERE c1 BETWEEN 10 and 20 FOR UPDATE;，为防止其他事务将值15插入到t.c1列中，无论该列中是否已有这样的值，该范围中所有现有值之间的间隙都会被锁定。
Gap Lock是在性能和并发能力上的一种权衡方案，仅在部分事务隔离级别上采用
SELECT * FROM child WHERE id = 100;对这类id具有唯一索引的情况，只会明确加Record锁，而不会加Gap锁
createtable t(a int primary key); insertinto t select1; insertinto t select2; insertinto t select5;
此时插入按唯一键a插入时，只会锁定单条记录，如下表：
实例2：辅助索引（非唯一索引或主键索引）
1 2 3 4 5 6
createtable z(a int, b int, primary key(a), key(b)); insertinto z select1,1; insertinto z select3,1; insertinto z select5,3; insertinto z select7,6; insertinto z select10,8;
执行SQLselect * from z where b=3 for update时，锁处理为：
存在两个索引，需分别进行锁定
对于a聚簇索引，对a=5进行RecordLock
对于b辅助索引，按Next-Key对区间(1,3]加锁
同时，InnoDB还会对下一个键值加GapLock，即对区间(3,6]加锁
其他会话的下列SQL将被阻塞（锁等待）：
1 2 3
select * from z where a=5forupdate; -- a=5，是`a=5`的RecordLock锁定记录，锁等待 insertinto z select4,2; -- b=2，在锁定区间(1,3]范围内，锁等待 insertinto z select6,5; -- b=5 在锁定区间(3,6]范围内，锁等待
for _, v := range data { if v.F1 == 1 { v = process(v) if err := v.Call(); err == nil { v.Send() } else { return err } } else { log.Printf("Invalid v: %v", v) } }
1 2 3 4 5 6 7 8 9 10 11 12
for _, v := range data { if v.F1 != 1 { log.Printf("Invalid v: %v", v) continue } v = process(v) if err := v.Call(); err != nil { return err } v.Send() }
不必要的 else
如果在 if 的两个分支中都设置同样的变量，则可以用单个 if 替换它。
Bad
Good
1 2 3 4 5 6
var a int if b { a = 100 } else { a = 10 }
1 2 3 4
a := 10 if b { a = 100 }
顶层变量的声明
在顶层，使用标准的 var 关键字。不要指定类型，除非它与表达式的类型不同。
Bad
Good
1 2 3
var _s string = F()
funcF()string { return"A" }
1 2 3 4
var _s = F() // F 已经声明了返回一个 string，我们不需要再次指定类型
funcF()string { return"A" }
如果表达式的类型与请求的类型不完全匹配，请指定类型。
1 2 3 4 5 6 7 8
type myError struct{}
func(myError)Error()string { return"error" }
funcF()myError { return myError{} }
var _e error = F() // F 返回了一个 myError 类型的对象，但是我们想要 error
funcreader(c chanint, m *sync.RWMutex, wg *sync.WaitGroup) { sleep() m.RLock() c <- 1 sleep() c <- -1 m.RUnlock() wg.Done() }
funcwriter(c chanint, m *sync.RWMutex, wg *sync.WaitGroup) { sleep() m.Lock() c <- 1 sleep() c <- -1 m.Unlock() wg.Done() }
funcmain() { var m sync.RWMutex var rs, ws int rsCh := make(chanint) wsCh := make(chanint) gofunc() { for { select { case n := <-rsCh: rs += n case n := <-wsCh: ws += n } fmt.Printf("%s%s\n", strings.Repeat("R", rs), strings.Repeat("W", ws)) } }() wg := sync.WaitGroup{} for i := 0; i < 10; i++ { wg.Add(1) go reader(rsCh, &m, &wg) } for i := 0; i < 3; i++ { wg.Add(1) go writer(wsCh, &m, &wg) } wg.Wait() }
func(rw *RWMutex)RUnlock() { ... if r := atomic.AddInt32(&rw.readerCount, -1); r < 0 { if r+1 == 0 || r+1 == -rwmutexMaxreader { race.Enable() thrSw("sync: RUnlock of unlocked RWMutex") } // A writer is pending. if atomic.AddInt32(&rw.readerWait, -1) == 0 { // The last reader unblocks the writer. runtime_Semrelease(&rw.WriteSem, false) } } ... }
func(rw *RWMutex)Unlock() { ... r := atomic.AddInt32(&rw.readerCount, rwmutexMaxreader) if r >= rwmutexMaxreader { race.Enable() throw("sync: Unlock of unlocked RWMutex") } for i := 0; i < int(r); i++ { runtime_Semrelease(&rw.readerSem, false) } rw.w.Unlock() ... }
]]><blockquote>
<p> There are only two hard things in Computer Science: cache invalidation and naming things.<br> 计算机科学中有两件难事：缓存失效和命名<br> – Phil Karlton</p>
</blockquote>
<p>From <a href="https://martinfowler.com/bliki/TwoHardThings.html" target="_blank" rel="noopener">Martin Fowler : TwoHardThings</a></p>
<p>缓存系统一定程度上极大提升系统并发能力，但同样也增加额外技术考虑因素，下面针对缓存系统设计与使用中面临的常见问题展开。</p>
<ul>
<li>缓存应用的典型场景</li>
<li>缓存雪崩</li>
<li>缓存穿透</li>
<li>缓存更新与数据一致性解析MySQL事务隔离级别http://fivezh.github.io/2019/02/01/MySQL-Transaction-Isolation-Level/2019-02-01T03:22:26.000Z2020-01-31T14:13:16.000Z事务隔离是分布式系统对一致性保证的重要机制，是保证ACID的重要基础设施。MySQL InnoDB事务隔离级别官方说明中对MySQL的事务隔离机制有详细介绍。
注意：根据ANSI SQL Standard在可重复读级别下允许出现幻读，MySQL实现满足标准，这不是Bug。但PostgreSQL在可重复读时不会出现幻读问题，这是不同引擎实现机制上的差异。 By default, InnoDB operates in REPEATABLE READ transaction isolation level. In this case, InnoDB uses next-key locks for searches and index scans, which prevents phantom rows (see Section 15.7.4, “Phantom Rows”). 缺省的，MySQL采用REPEATABLE READ事务隔离级别，采用Next-key锁机制来避免幻读问题
intfreeMemoryIfNeeded(void){ while (mem_freed < mem_tofree) { if (server.maxmemory_policy == REDIS_MAXMEMORY_NO_EVICTION) return REDIS_ERR; /* We need to free memory, but policy forbids. */
if (server.maxmemory_policy == REDIS_MAXMEMORY_ALLKEYS_LRU || server.maxmemory_policy == REDIS_MAXMEMORY_ALLKEYS_RANDOM) {......} /* volatile-random and allkeys-random policy */ if (server.maxmemory_policy == REDIS_MAXMEMORY_ALLKEYS_RANDOM || server.maxmemory_policy == REDIS_MAXMEMORY_VOLATILE_RANDOM) {......} /* volatile-lru and allkeys-lru policy */ elseif (server.maxmemory_policy == REDIS_MAXMEMORY_ALLKEYS_LRU || server.maxmemory_policy == REDIS_MAXMEMORY_VOLATILE_LRU) { // 淘汰池函数 evictionPoolPopulate(dict, db->dict, db->eviction_pool); while(bestkey == NULL) { evictionPoolPopulate(dict, db->dict, db->eviction_pool); // 从后向前逐一淘汰 for (k = REDIS_EVICTION_POOL_SIZE-1; k >= 0; k--) { if (pool[k].key == NULL) continue; de = dictFind(dict,pool[k].key); // 定位目标
/* Remove the entry from the pool. */ sdsfree(pool[k].key); /* Shift all elements on its right to left. */ memmove(pool+k,pool+k+1, sizeof(pool[0])*(REDIS_EVICTION_POOL_SIZE-k-1)); /* Clear the element on the right which is empty * since we shifted one position to the left. */ pool[REDIS_EVICTION_POOL_SIZE-1].key = NULL; pool[REDIS_EVICTION_POOL_SIZE-1].idle = 0;
/* If the key exists, is our pick. Otherwise it is * a ghost and we need to try the next element. */ if (de) { bestkey = dictGetKey(de); // 确定删除键 break; } else { /* Ghost... */ continue; } } } } /* volatile-ttl */ elseif (server.maxmemory_policy == EDIS_MAXMEMORY_VOLATILE_TTL) {......}
The decay time is the obvious one, it is the amount of minutes a counter should be decayed, when sampled and found to be older than that value. A special value of 0 means: always decay the counter every time is scanned, and is rarely useful. The counter logarithm factor changes how many hits are needed in order to saturate the frequency counter, which is just in the range 0-255. The higher the factor, the more accesses are needed in order to reach the maximum. The lower the factor, the better is the resolution of the counter for low accesses
composer install错误信息如下he requested package could not be found in any version, there may be a typo in the package name.
可能的问题：
包名错误
未找到满足要求的代码库版本
解决：composer依赖tag信息选择版本，代码库必须有对应tag信息
git tag -a v1.0.0 -m "initial release"
1 2 3 4 5 6 7 8 9
Your requirements could not be resolved to an installable set of packages.
Problem 1 - The requested package rrc/c2b_utils could not be found in any version, there may be a typo in the package name.
Potential causes: - A typo in the package name - The package is not available in a stable-enough version according to your minimum-stability setting see <https://getcomposer.org/doc/04-schema.md#minimum-stability> for more details.