fast query inexplicably times out sometimes. #2865

Describe the bug

running the query via REST on an unchanged dataset sometimes fails--reportedly due to timeout. however, a reboot seems to fix it. 100% of attempts so far against the web interface have correctly returned the result of the query.

the "timeout" is set in quest conf to 1200 seconds, but when the query timesout, it fails instantly (not even the default of 60 seconds elapses or anything close. would guess at failure <1sec)

when the query doesn't fail, it takes about 600ms first run, then 200ms thereafter.

To reproduce

difficult to be sure.

1. attempt to run query similar to "Select distinct Expirationdate from TABLE where THIS_SYMBOL = 'blah' ;"
2. run random ingestion and retrieval queries against db that do not add data to TABLE
3. re-run the query
4. go to 2 while not crash

Expected Behavior

no time out.

Environment

- **QuestDB version**: 6.6.1
- **OS**: win11
- **Browser**: Edge

Additional context

log of failed query

2022-12-13T18:26:31.233981Z I i.q.c.p.WriterPool closed [table=PJG, reason=IDLE, by=35]
2022-12-13T20:18:50.406730Z I http-server connected [ip=127.0.0.1, fd=5076]
2022-12-13T20:18:50.409218Z I i.q.c.h.p.QueryCache miss [thread=questdb-shared-8, sql=select distinct TradeDate from PJ where RICRoot = 'SPX.U']
2022-12-13T20:18:50.415254Z I i.q.c.p.AbstractMultiTenantPool open 'PJ' [at=0:0]
2022-12-13T20:18:50.424758Z I i.q.g.SqlCompiler plan [q=select-distinct [TradeDate] TradeDate from (select-choose [TradeDate] TradeDate from (select [TradeDate, RICRoot] from PJ timestamp (TradeDate) where RICRoot = 'SPX.U')), fd=5076]
2022-12-13T20:18:50.439592Z I i.q.c.h.p.TextQueryProcessor [5076] execute-new [q=select distinct TradeDate from PJ where RICRoot = 'SPX.U', skip: 0, stop: 9223372036854775807]
2022-12-13T20:18:50.440644Z I i.q.c.TableReader open partition C:\Users\bruvn\QSP\bin\questdb\bin\qdbroot\db\PJ\2005 [rowCount=842, partitionNameTxn=-1, transientRowCount=6893530, partitionIndex=0, partitionCount=18]
2022-12-13T20:18:50.443630Z I i.q.c.TableReader open partition C:\Users\bruvn\QSP\bin\questdb\bin\qdbroot\db\PJ\2006 [rowCount=1281, partitionNameTxn=-1, transientRowCount=6893530, partitionIndex=1, partitionCount=18]
2022-12-13T20:18:50.445326Z I i.q.c.TableReader open partition C:\Users\bruvn\QSP\bin\questdb\bin\qdbroot\db\PJ\2007 [rowCount=1273, partitionNameTxn=-1, transientRowCount=6893530, partitionIndex=2, partitionCount=18]
2022-12-13T20:18:50.446755Z I i.q.c.TableReader open partition C:\Users\bruvn\QSP\bin\questdb\bin\qdbroot\db\PJ\2008 [rowCount=1265, partitionNameTxn=-1, transientRowCount=6893530, partitionIndex=3, partitionCount=18]
2022-12-13T20:18:50.448014Z I i.q.c.TableReader open partition C:\Users\bruvn\QSP\bin\questdb\bin\qdbroot\db\PJ\2009 [rowCount=1260, partitionNameTxn=-1, transientRowCount=6893530, partitionIndex=4, partitionCount=18]
2022-12-13T20:18:50.449437Z I i.q.c.TableReader open partition C:\Users\bruvn\QSP\bin\questdb\bin\qdbroot\db\PJ\2010 [rowCount=1260, partitionNameTxn=-1, transientRowCount=6893530, partitionIndex=5, partitionCount=18]
2022-12-13T20:18:50.450644Z I i.q.c.TableReader open partition C:\Users\bruvn\QSP\bin\questdb\bin\qdbroot\db\PJ\2011 [rowCount=1260, partitionNameTxn=-1, transientRowCount=6893530, partitionIndex=6, partitionCount=18]
2022-12-13T20:18:50.452014Z I i.q.c.TableReader open partition C:\Users\bruvn\QSP\bin\questdb\bin\qdbroot\db\PJ\2012 [rowCount=1250, partitionNameTxn=-1, transientRowCount=6893530, partitionIndex=7, partitionCount=18]
2022-12-13T20:18:50.454007Z I i.q.c.TableReader open partition C:\Users\bruvn\QSP\bin\questdb\bin\qdbroot\db\PJ\2013 [rowCount=1261, partitionNameTxn=-1, transientRowCount=6893530, partitionIndex=8, partitionCount=18]
2022-12-13T20:18:50.455145Z I i.q.c.TableReader open partition C:\Users\bruvn\QSP\bin\questdb\bin\qdbroot\db\PJ\2014 [rowCount=1263, partitionNameTxn=-1, transientRowCount=6893530, partitionIndex=9, partitionCount=18]
2022-12-13T20:18:50.456494Z I i.q.c.TableReader open partition C:\Users\bruvn\QSP\bin\questdb\bin\qdbroot\db\PJ\2015 [rowCount=1266, partitionNameTxn=-1, transientRowCount=6893530, partitionIndex=10, partitionCount=18]
2022-12-13T20:18:50.457669Z I i.q.c.TableReader open partition C:\Users\bruvn\QSP\bin\questdb\bin\qdbroot\db\PJ\2016 [rowCount=1268, partitionNameTxn=-1, transientRowCount=6893530, partitionIndex=11, partitionCount=18]
2022-12-13T20:18:50.458764Z I i.q.c.TableReader open partition C:\Users\bruvn\QSP\bin\questdb\bin\qdbroot\db\PJ\2017 [rowCount=2651512, partitionNameTxn=-1, transientRowCount=6893530, partitionIndex=12, partitionCount=18]
2022-12-13T20:18:50.576731Z I i.q.c.TableReader open partition C:\Users\bruvn\QSP\bin\questdb\bin\qdbroot\db\PJ\2018 [rowCount=3525251, partitionNameTxn=-1, transientRowCount=6893530, partitionIndex=13, partitionCount=18]
2022-12-13T20:18:50.728681Z I i.q.c.TableReader open partition C:\Users\bruvn\QSP\bin\questdb\bin\qdbroot\db\PJ\2019 [rowCount=3751935, partitionNameTxn=-1, transientRowCount=6893530, partitionIndex=14, partitionCount=18]
2022-12-13T20:18:50.790935Z C i.q.c.h.p.TextQueryProcessor [5076] Server error executing query select distinct TradeDate from PJ where RICRoot = 'SPX.U'
io.questdb.cairo.CairoException: [-1] timeout, query aborted [fd=5076]

2022-12-13T20:18:50.816649Z I http-server scheduling disconnect [fd=5076, reason=12]
2022-12-13T20:18:50.816662Z I http-server disconnected [ip=127.0.0.1, fd=5076, src=queue]
2022-12-13T20:18:50.820415Z I http-server connected [ip=127.0.0.1, fd=4700]
2022-12-13T20:18:50.820761Z I i.q.c.h.p.QueryCache miss [thread=questdb-shared-8, sql=select distinct ExpirationDate from PJ where RICRoot = 'SPX.U']
2022-12-13T20:18:50.820985Z I i.q.g.SqlCompiler plan [q=select-distinct [ExpirationDate] ExpirationDate from (select-choose [ExpirationDate] ExpirationDate from (select [ExpirationDate, RICRoot] from PJ timestamp (TradeDate) where RICRoot = 'SPX.U')), fd=4700]
2022-12-13T20:18:50.826220Z I i.q.c.h.p.TextQueryProcessor [4700] execute-new [q=select distinct ExpirationDate from PJ where RICRoot = 'SPX.U', skip: 0, stop: 9223372036854775807]
2022-12-13T20:18:50.924579Z C i.q.c.h.p.TextQueryProcessor [4700] Server error executing query select distinct ExpirationDate from PJ where RICRoot = 'SPX.U'
io.questdb.cairo.CairoException: [-1] timeout, query aborted [fd=4700]

2022-12-13T20:18:50.925724Z I http-server scheduling disconnect [fd=4700, reason=12]
2022-12-13T20:18:50.925741Z I http-server disconnected [ip=127.0.0.1, fd=4700, src=queue]
2022-12-13T20:18:55.867258Z I http-server connected [ip=127.0.0.1, fd=964]
2022-12-13T20:18:55.867833Z I i.q.c.h.p.QueryCache miss [thread=questdb-shared-3, sql=select distinct TradeDate from PJ where RICRoot = 'SPX.U']
2022-12-13T20:18:55.868236Z I i.q.g.SqlCompiler plan [q=select-distinct [TradeDate] TradeDate from (select-choose [TradeDate] TradeDate from (select [TradeDate, RICRoot] from PJ timestamp (TradeDate) where RICRoot = 'SPX.U')), fd=964]
2022-12-13T20:18:55.868646Z I i.q.c.h.p.TextQueryProcessor [964] execute-new [q=select distinct TradeDate from PJ where RICRoot = 'SPX.U', skip: 0, stop: 9223372036854775807]
2022-12-13T20:18:55.930592Z C i.q.c.h.p.TextQueryProcessor [964] Server error executing query select distinct TradeDate from PJ where RICRoot = 'SPX.U'
io.questdb.cairo.CairoException: [-1] timeout, query aborted [fd=964]

2022-12-13T20:18:55.931485Z I http-server scheduling disconnect [fd=964, reason=12]
2022-12-13T20:18:55.931496Z I http-server disconnected [ip=127.0.0.1, fd=964, src=queue]
2022-12-13T20:18:55.940627Z I http-server connected [ip=127.0.0.1, fd=5012]
2022-12-13T20:18:55.941116Z I i.q.c.h.p.QueryCache miss [thread=questdb-shared-3, sql=select distinct ExpirationDate from PJ where RICRoot = 'SPX.U']
2022-12-13T20:18:55.941441Z I i.q.g.SqlCompiler plan [q=select-distinct [ExpirationDate] ExpirationDate from (select-choose [ExpirationDate] ExpirationDate from (select [ExpirationDate, RICRoot] from PJ timestamp (TradeDate) where RICRoot = 'SPX.U')), fd=5012]
2022-12-13T20:18:55.951299Z I i.q.c.h.p.TextQueryProcessor [5012] execute-new [q=select distinct ExpirationDate from PJ where RICRoot = 'SPX.U', skip: 0, stop: 9223372036854775807]
2022-12-13T20:18:56.006621Z C i.q.c.h.p.TextQueryProcessor [5012] Server error executing query select distinct ExpirationDate from PJ where RICRoot = 'SPX.U'
io.questdb.cairo.CairoException: [-1] timeout, query aborted [fd=5012]

2022-12-13T20:18:56.008490Z I http-server scheduling disconnect [fd=5012, reason=12]
2022-12-13T20:18:56.008521Z I http-server disconnected [ip=127.0.0.1, fd=5012, src=queue]