FPGA Mining Guide 08/2018

Thomas Block
https://twitter.com/ThomasBlock3
https://www.youtube.com/channel/UCCTIyxOWqhwB3cu7PbRIgQA

Google Sheet Version (for Comments):

https://docs.google.com/document/d/1LamZtUdye18UJ_5vewdd38UBdk1c7LC_dsu5ZHdC2
gg/edit?usp=sharing

Xilinx DK-U1-VCU1525-P-G
Xilinx DK-U1-VCU1525-P-G
Introduction
VCU1525 specs
How to get it
RAM
Voucher for software design toolkit
DC1613A
what?
why?
how?
Hardware Setup
Cooling
Hardware Connections
PCIE Riser Power
PCIE Riser USB to mainboard
Mini-USB from card to mainboard
8 port PCIE power connection
LT powerplay device
Second Pc
Board LEDs
Using Vivado Labs
Using LT Powerplay Software
Mining 0xBTC
clocks
Mining Tribus
Mining Keccak
Profit Estimation
Real World income
Next Steps
Immersion Cooling
Theory
Coolant
Heat Removal
my immersion cooling setup
Real World income
next step
Introduction

Hi. In this guide i am constantly collecting all the information i learned about my FPGA
cryptocurrency mining experience in 2018. It may be useful for people who want to get into
fpga mining.

read, learn, share your knowledge.

if you think that something is unclear or missing, just contact me and i will update it.

Donations go here (btc):

bc1q7lryh7h4qnpgtjjq0pz70wwnkyuaj5lg2mlml7
3DbZzhr6mc2NWZuEY6Fic51SYZ8yqkn48Z

Thomas Block

This Work is based on the hard work of the FPGA developer scene. I appreciate their vision
of sharing bitstreams very much. Thank you! Here are some links to their original posts:

whitefire990
https://bitcointalk.org/index.php?topic=3459858.0
http://zetheron.com/

senseless
https://bitcointalk.org/index.php?topic=3688965.0
https://fpga.land/

GPUHoarder
https://bitcointalk.org/index.php?topic=4391318.0
http://www.squirrelsresearch.com/

DigitalCruncher
https://bitcointalk.org/index.php?action=profile;u=1735391
https://github.com/digitalcruncher

and many more, join us on:

https://discord.gg/25DpDCS

there are also some other services:

https://store.mineority.io

https://fpgatalk.org/
https://fpga.guide/en/

https://nocroom.com/fpga
VCU1525 specs
https://www.xilinx.com/products/boards-and-kits/vcu1525-a.html
manual:
https://japan.xilinx.com/support/documentation/boards_and_kits/vcu1525/ug1268-vcu1525-r
econfig-accel-platform.pdf

How to get it

once listed directly on ​https://www.avnet.com/shop/emea/​ now redireted to EBV Elektronik:

https://www.avnet.com/wps/portal/ebv/products/new-products/npi/2018/xilinx-vcu1525-accel
eration-kit

as of August 2018, you can order it from AVNet USA directly. They get it from Asia, so lead
time is 4-8 weeks. They can then express ship it worldwide ( except Cuba, Iran, North Korea,
Sudan and Syria ). Price is 4995 USD plus local taxes. Worked well for me in germany.

There also was a group buy of 5000 units on ​https://fpga.land/​ - but it is sold out and may
also be delayed. so you might be faster by ordering directly. the downside is that you only
get that passive cooled version which needs a lot of work for proper cooling. the group
variant called BCU1525 has already better cooling installed.

ALTERNATIVES 10/2018:
https://talentpros.com.tw/%E6%9C%8D%E5%8B%99%E8%88%87%E7%94%A2%E5%93
%81%E4%BB%8B%E7%B4%B9

https://shop.fpga.guide/collections/all/products/btu9p-by-tul

RAM
the card comes with 4 * 16 GB RAM, but actual bitstreams don’t need it. so you can remove
and even sell it. ( but maybe future software will take profit from / need RAM )

Voucher for software design toolkit

it is included in the box. you don’t need it for mining. it says it is valid one year. so maybe
you can sell it / resell it when selling the card.
DC1613A

what?
http://www.analog.com/media/en/dsp-documentation/evaluation-kit-manuals/dc1613af.pdf
this additional LT powerplay device can undervolt the card. there are two modes of
operation:
● buy one unit, and undervolt each card to 0.72 V
● buy one unit per fpga and control the voltages live while mining ( you can go even
lower, but it can crash )

why?
the stock value of 0.85V causes a really high power usage - i could not even use the slowest
bitstream. with this device i can go to much higher speed, at lower temperatures. see my
measurements of power consumption at the wall:

how?
you have to buy the device and build your own connector, as described here:
http://zetheron.com/Downloads/VCU1525_DC1613A_Adapter.pdf
The Software usage is described below.
Hardware Setup

my advice is to use PCIE risers, known from GPU mining: so you can locate the card on a
place where you can cool it easily ( with fans e.g. ). and you can be sure you don’t draw too
much power from the mainboard.

choose a cpu with >0,5 cores per fpga.

choose a mainboard with 1 pcie slot per fpga. ( 1x is fine )
choose a quality PSU ( 400 Watt per FPGA, each needs an 8 port pcie power and a riser
power )
4 GB RAM is ok
use a ssd.
create a dual boot system with windows and linux - so you can run software for either
operating system.
Cooling
the passive cooled version i got needs a lot of cooling. i experimented with HT water pipes.
they are cheap and made for up to 95°C. connected with two antminer fans you get a lot of
airflow: ( the mining bazooka )
But i also used industrial fans and pvc air pipes:
Antminer Cooling:
Hardware Connections

to get the best results, the card needs FIVE connections.

PCIE Riser Power

from psu to riser. the card draws 75W power from here. ( maybe even more? see below )

PCIE Riser USB to mainboard

some bitstreams communicate over this bus with the card. ( DigitalCruncher )

Mini-USB from card to mainboard

Vivado needs this connection to program the bitstream. some bitstreams communicate over
this bus with the card. ( Whitefire ).

8 port PCIE power connection

from psu to card. PCIE specs allow 150 Watt. but this card can draw 300W. so don’t use a
cheap psu.

LT powerplay device
connect your card to the LT device. ( but the usb connection from LT goes to a second pc )

Second Pc
you will need a very basic windows pc with usb ( old laptop ). we will connect it to the LT
device - so we can control the telemetry of the cards regardless in which state / operating
system the first pc is. and usb ports won’t interfere.
Board LEDs

the board has 5 LEDs. my actual guess of assignment:

● blue light at the top glowing = bitstream is programmed
● red light below glowing: card is in error state -> check LT Powerplay, reset
● three lights in a row
○ green blinking = working
○ yellow glowing = pcie power is present
○ red: ( not seen yet )
Using Vivado Labs
● register for free, then you can download the free “vivado lab edition”
● https://www.xilinx.com/support/download.html
● you can use Windows or linux
if everything works well you can see your boards. if not, the usb connection might be
missing, or the board could be in error state (“[Labtools 27-2269] No devices detected on
target localhost:3121/xilinx_tcf/Xilinx/12809621t006A. Check cable connectivity and that the
target board is powered up then use the disconnect_hw_server and connect_hw_server to
re-register this hardware target.“)
Using LT Powerplay Software
also check zetherons manual:
http://zetheron.com/Downloads/VCU1525_DC1613A_Adapter.pdf

install:
● .NET 3.5 ​https://www.microsoft.com/de-de/download/details.aspx?id=21
● USB port driver ​ ​http://www.ftdichip.com/Drivers/D2XX.htm
● LT Power Play ​http://www.ltpowerplay.com/download/

open the LT power play. if it is requesting a licence, communication with the card does not
work. click “detect chips”

problem: you can not use several LT devices on one pc. the data of all connected devices
will mix up. so we will only connect one at the time. but you can keep the software running:
just change usb cables and it will update.

If you start the software and see this, you usb connection to the box is not correct:

you should see this and click “detect chips”

working software could look like this ( i closed some subwindows ):

now you can change the voltage and save it:

Sometimes it happens that you want to save something and error “registers were not written”
occurs. Just press 3 times OK and try it again:

if the card is in error state, you have to click the clear button and maybe reset.
Mining 0xBTC
you have several fundamental options:

● mine in a pool
○ easy setup
○ high fees
● mine solo
○ no fees
○ you need a ETH private key with balance on it
● mine 0xBTC
○ high long term stability
○ high difficulty: solo mining can take up to 10 days for one block
● mine 0x Altcoins
○ low difficulty: solo-mine one Block per hour
○ most of them don’t even have an exchange listing - could be completely
worthless

you need Zetherons Miner, read his great instructions

http://zetheron.com/Downloads/VCU1525_Instructions.pdf
http://zetheron.com/index.php/downloads/

● http://www.ftdichip.com/Drivers/CDM/CDM21228_Setup.zip
● https://www.microsoft.com/en-ca/download/details.aspx?id=40784
● unzip miner software

for pool mining

● edit tokenminer.ini
○ MinerAcct=0x00000yourpublickey
● fx-tokenminer-v1.32 -C -P

for solo mining

● create an ethereum private key ( ​https://www.myetherwallet.com/​ )
● send some ether ( 10 USD ) to the corresponding public key ( you need gas to create
blocks )
● request a free api key on ​https://infura.io/
● edit tokenminer.ini
○ Host=https://mainnet.infura.io/<YOUR_API_KEY>
○ RPCPort=443
○ MinerAcct=0x00000yourpublickey
○ AcctPK=00000yourprivatekey
● fx-tokenminer-v1.32 -C -S
● send your coins to another address after mining ( as the private key lies on your pc
around )

this is the output when everything is ok:

when the mining crashes, two things happen
● the power usage goes down ( 200W -> 20W )
● there are no more “Hash” outputs, only “Block” outputs:
when this happens and you restart the software, it does no longer work:

you then have to reprogram the bitstream. To do so you also have to increase voltage back
to 0.72V. So you certainly want to prevent this - very annoying.
sometimes this happens: “Max Nonce: Failed to write new vector to fpga” ( ~48 hours for
me)

create an endless loop batch file to restart after errors like above:

You should put a shortcut in windows autostart ( several batchfiles for multiple cards ) - so it
will continue mining after reboot / windows update ( dont’t forget to choose windows as
standard in multi-boot systems )
clocks
Bitstream V error rate Power@Wall
[W]

idle 0

programming 20

11.5 GH 0.72 0% 210

11.5 GH 0.69 0% 195

11.5 GH 0.66 0% 180

11.5 GH 0.64 0% 170

11.5 GH 0.63 0,2 % 160 <- Optimum?

11.5 GH 0.62 1% 155

11.5 GH 0.61 3% 150

13 GH 0.72 0% 240

13 GH 0.69 0% 220

13 GH 0.66 0% 205

13 GH 0.64 0.1 % 195

13 GH 0.63 0.5 % 185 <- Optimum?

13 GH 0.62 3% 180

13 GH 0.61 6% 175

there are 2 types of errors:

● board is still hashing, but error rate increases: this value can be tuned here
● board is crashed. for me it crashed much less on 11.5 GH

summary:

Pro Contra

0.85 V No Dongle necessary. no crashes very hot card

0.72 V quite stable and cold

0.62 V very cool much interaction needed, reprogramming

a word about infura.io :
sometimes is see connection errors ( libcurl ), so we will sometimes loose blockchain data,
maybe can not submit or get stuck at donation mining ( pool )
this is nothing unusual on a free cloud service. for best results ( especially with many cards ),
you should set up a etherum full node! so you have always a reliable connection ( and may
even reduce orphan blocks )

you have to install a ethereum node and open the rpc port to the whole internet ( be careful -
no password , so don't hold funds in that node )
here is my docker command line which would do this:
docker run --restart=unless-stopped -d -v /path/to/your/data/storage:/root -p 8545:8545 -p
30303:30303 ethereum/client-go --rpc --rpcaddr 0.0.0.0 --rpccorsdomain "*" --syncmode full

Air cooling comparison: 10 GH, 0.73V ( ~ 180W )

°C Miner Temp1 Temp2

high Airflow 68 71 80

low Airflow 81 84 90

0xToken Altcoins ( 13.09.2018 )

S = 10 GH/s for simplicity

Income = S * 3600 * 24 *30 / ( Diff * 2^22 Hashes ) * Reward * Price

http://www.wolframalpha.com/input/?i=3600*24*30%2F+(+864881523+*+2%5E22+)+*+1000
0000000+*+50+*+0.00314+*+190

but we have to pay an ETH network fee for every created block:

Tx fee = Blocks * 0,1 $ = S * 3600 * 24 *30 / ( Diff * 2^22 Hashes ) * 0,1 $

http://www.wolframalpha.com/input/?i=3600*24*30%2F+(+864881523+*+2%5E22+)+*+1000
0000000+*+0.1

1 ETH = 190 USD

Diff Minimum = 100% Attack = 1 Block in 10 Minutes: S * 600s / 2^22 = 1430511

Coin Diff Reward Price Exchange $/Month $ Fee

[ETH]

0xBTC 864881523 50 0.00314 Mercatox 213 -0.7

 Atlantis 472078 500 0.0000000 enclaves.io 0.4 -432
1430511 01

0xDiary 1 4 0 - 0 -432
1430511

0xSkorch 63160 50 0 - 0 -432

1430511

Kiwi 18416 5000 0.0000000 enclaves.io 63 -432

1430511 155

0xLTC 28 50 0 -
1430511

0xZIBI 0 -

0xDOGEC 0
OIN

0xRamen 0

0xbcash 0

None of these Altcoins does make any sense. All devs left. Just all are 0xbtc copy scams.
You could mine like 95% of each coin - but you would have to pay around 400 Dollar in
Etherum Network fees. The following Table will proof that

Token Tx of whole Contract usage

0xBTC 185000 medium

Atlantis 31080 low

0xDiary 7533 low

0xSkorch 10457 low

0xLTC 38443 no, last 60 days without tx

Kiwi 114098 medium

0xZibi 2901 no

0xDOGE 14155 no, last 80 days without tx

Ramen 2112 no

0xbcash 65283 low

Mining Tribus

This needs Linux.

- install Windows 10 ( for dual boot, half disk size )
- install Linux Mint ( Rest of the Disk )
- install updates
- install teamviewer .deb ( start with system, lan only, define password, no random
password)
- install vivado lab edition
- download ( free account needed)
https://www.xilinx.com/member/forms/download/xef.html?filename=Xilinx_Viv
ado_Lab_Lin_2018.2_0614_1954.tar.gz
- sudo ./xsetup
- install usb drivers
- cd
/opt/Xilinx/Vivado_Lab/2018.2/data/xicom/cable_drivers/lin64/install_script/ins
tall_drivers
- sudo ./install_drivers
- replug cables
- set usb permissions for “user123”
- sudo gpasswd --add user123 dialout
- start vivado
- source /opt/Xilinx/Vivado_Lab/2018.2/settings64.sh
- vivado_lab &
- open hardware manager ( see other part of manual )
- program bitstream ( vcu1525_tribus_3_650.bit )
- reboot system ( important )
- download miner
- https://github.com/digitalcruncher/keccak-tribus
- extract
- cd vcu1525-keccak-tribus-variable/ccminer
- sudo apt-get install libcurl3 -y
- edit tribus.sh
- export LD_LIBRARY_PATH=./
- echo 1 > /sys/bus/pci/devices/0000:03:00.0/enable
- cp ./license.tribus ./license.txt
- ./activator
- sleep 1
- ./ccminer-keccak-tribus -a tribus -o stratum+tcp://yiimp.eu:8533 -u
DF8V3HXmWgs9WrvJpCKVK18zm69HsAtLKP -p c=DNR
- edit cores.txt
- 1 ( set higher for more speed -> needs more cooling and a licence )
- edit devices.txt
- /sys/bus/pci/devices/0000:03:00.0/resource0
- run
- sudo ./tribus.sh
how to find the correct pci port: ( replace /sys/bus/pci/devices/0000:03:00.0/resource0 )

lspci | grep Xilinx

02:00.0 Serial controller: Xilinx Corporation Device 6a90
02:00.1 Serial controller: Xilinx Corporation Device 6a8f
03:00.0 Serial controller: Xilinx Corporation Device 6a90
03:00.1 Serial controller: Xilinx Corporation Device 6a8f

-> try these four numbers until it works: 02:00.0, 02:00.1 , 03:00.0, 03:00.1

how to get licence.tribus?

-> contact DigitalCruncher. You will need it if you want to run more than 1 core.

errors?

./ccminer-keccak-tribus: /usr/lib/x86_64-linux-gnu/libcurl.so.4: version `CURL_OPENSSL_3'

not found (required by ./ccminer-keccak-tribus)

-> your linux uses libcurl4, whereas the binary needs libcurl3 ->
sudo add-apt-repository http://security.ubuntu.com/ubuntu xenial-security main
sudo apt-get install libcurl3

crash
-> do not run vivado and ccminer at the same time!
wokring miner:
not working miner:
here we see a problem with the activator:
for a valid activation, the “Status” after the line “unlock” should beginn with “0xC00” and not
“0x800” . so if you have a valid activation code, try the following
● remove activator from your sh-file
● run activator.sh manually ( edit for your pcie slot )
● run it several times until you have “0xC00”
● dont run it any more afterwards

Tribus power usage ( power hard to mease, jumps much )

cores GH V Power @Wall

1 0,6 0.85 105 W

1 0,6 0.72 75 W

1 0,6 0.62 no shares

2 1,2 0.75 160 W <- optimum

2 1,2 0.72 150 W

2 1,2 0.70 140 W less shares

2 1,2 0.65 no shares no shares

3 1,8 0.70 200 W

Tribus Altcoin Overview - Value of one Block

● 2 Denarius = 0,70 € ( several exchanges )
○ after 2500 Blocks you can create a masternode
● 1 BZL = 0,01 € ( Cryptobridge )
○ after 2000 Blocks you can create a masternode
● 22.4 Scriv = 0,02 € ( shady exchange )
○ after 4500 Blocks you can create a masternode
not recommended:
● 2 Zula = 0,001 € ( Cryptobridge, but dev silent = dead )
● 6 Virtus = 0,001 € ( Cryptobridge, but dev silent = dead )
● 5 Tincoin ( not listed, dev silent = dead )
● 300 Vegacoin ( long time dead )
● 0.8 Keyco ( POW ended )
● 0.05 Averopay ( not listed. it has 99,5% premine - absolute scam )
○ after 100000 Blocks you can create a masternode (wtf.. )

https://crypt0.zone/coins/tribus

conclusion: Only Denarius, BZL and Scriv are useful profit-switching coins. (exchangeable).
Only Denarius has long term potential.
You could mine Zula or Virtus and hope for a pump ( create now sellorders at Cryptobridge
with 10x of the actual price. check in 6 months )
important ccminer command line options
(normally visible via -h )

-f, --diff-factor Divide difficulty by this factor (default 1.0)

-m, --diff-multiplier Multiply difficulty by this value (default 1.0)
-o, --url=URL URL of mining server
-O, --userpass=U:P username:password pair for mining server
-u, --user=USERNAME username for mining server
-p, --pass=PASSWORD password for mining server
--cert=FILE certificate for mining server using SSL
-x, --proxy=[PROTOCOL://]HOST[:PORT] connect through a proxy
-t, --threads=N number of miner threads (default: number of nVidia GPUs in your
system)
-r, --retries=N number of times to retry if a network call fails
(default: retry indefinitely)
-R, --retry-pause=N time to pause between retries, in seconds (default: 15)
--shares-limit maximum shares to mine before exiting the program.
--time-limit maximum time [s] to mine before exiting the program.
-T, --timeout=N network timeout, in seconds (default: 300)
-s, --scantime=N upper bound on time spent scanning current work when
long polling is unavailable, in seconds (default: 5)
--submit-stale ignore stale job checks, may create more rejected shares
--no-gbt disable getblocktemplate support (height check in solo)
--no-longpoll disable X-Long-Polling support
--no-stratum disable X-Stratum support
-q, --quiet disable per-thread hashmeter output
--no-color disable colored output
-D, --debug enable debug output
-P, --protocol-dump verbose dump of protocol-level activities
-b, --api-bind=port IP:port for the miner API (default: 127.0.0.1:4068), 0 disabled
--api-remote Allow remote control, like pool switching, imply --api-allow=0/0
--api-allow=... IP/mask of the allowed api client(s), 0/0 for all
--max-temp=N Only mine if gpu temp is less than specified value
--max-rate=N[KMG] Only mine if net hashrate is less than specified value
--max-diff=N Only mine if net difficulty is less than specified value
--keep-clocks prevent reset clocks and/or power limit on exit
--hide-diff Hide submitted shares diff and net difficulty
-B, --background run the miner in the background
--benchmark run in offline benchmark mode
-c, --config=FILE load a JSON-format configuration file
can be from an url with the http:// prefix

-> maybe it is possible to define a second (backup) pool in config file?

-> otherwise, create a shell script where you start one ccminer after the other, with several
pools, with “-r 3”, as endless loop. ( so when a pool is down, it will be tried 3 times, then
ccminer closes )
ccminer api access

it is nice to monitor your farm in a central tool like awesome miner. so you can define in
ccminer “--api-bind 0.0.0.0:1080” to listen on port 1080. awesome miner could also connect
and shows hasrate. but then i had a bug: ccminer crashed with the comment:
[2018-08-18 08:57:58] Your system does not support CUDA 8.0 API!
so i guess, api is just not fully implemented in this software
Mining Keccak

https://crypt0.zone/coins/keccakc

Altcoins: Worth of one Block

● Keccak
○ 59 SmartCash = 2,80 €
○ 8 Maxcoin = 0,28 €
○ Slothcoin (SLOTH) – Inactive ( but pool: ​https://pool.atomminer.com/​ )
○ 365Coin (365) – Inactive
○ Galleon (GLN) – Inactive
○ Helix Coin (HXC) – Inactive
○ CryptoMeth (METH) – Inactive
○ BitcointalkCoin (TALK) – Inactive ( but Pool: ​https://pool.atomminer.com/​ )
○ XDNA?
● Keccak-C
○ 34 Creativecoin = 1,35 €
Mining Zenprotocol

!= Zencash ( aka Horizen, equihash )

https://www.zenprotocol.com/en
https://coinmarketcap.com/currencies/zen-protocol/

new miner from 21.09.2018:

download binary and bitstream here:

https://github.com/digitalcruncher/zen-protocol

Altcoins: Worth of one Block

● Keccak-ZP
○ 50 ZP = 25 $

Explorer:
https://zp.io/

Pools:
https://zp.leafpool.com/​ -> has 95% of Network. 2% fee
https://zp.pandapool.pw/​ -> down
https://pool.atomminer.com/​ -> “red” ?

Profit Estimation:
Network: 8000 GH
1 VCU: 12 GH
100 Blocks = 6,5 h
24h = 400 Blocks
1 VCU = 400/8000*12 Blocks = 0,6 Blocks
VCU: 15$ / day

Leafpool:
Stratum Hostname/Location:
US: zp-us-east.leafpool.com
EU: zp-eu.leafpool.com
Asia: zp-asia.leafpool.com

Stratum Ports:
GPU Mining: 8811
Rigs/Mining Farm: 8822

NVIDIA (CUDA) Miners

ccminer-zp Github Download Link
ccminer -o stratum+tcp://zp-us-east.leafpool.com:8811 -u
YOUR_ADDRESS.WORKER_NAME -p x

./ccminer-zp -o stratum+tcp://zp-eu.leafpool.com:8822 -u
zen1q9xu2lxr55hucs36vc5vmydqj04y9vdgp3zx5n94l0mtwn9a2cs4sarn3t2.yourDonationNa
me -p x -q

Mining Zenprotocol

Cores Speed Voltage Power T ( Immersion )

21 12 GH 0.73 V 280W 58 / 68 / 73 °C

Solo Mining

Why? Leafpool is doing 100% attack on network. No Fee.

● install node: ​https://docs.zenprotocol.com/preparation

● point miner at own node: 127.0.0.1:11567 (? testing..)
Profit Estimation

last update: 11.08.2018

Coin Bitstream Hashrate Voltage power[W] blocks/day $ / month

0xBTC Zetheron 13 GH 0,62 160 0,25 200

Denarius DigitalCruncher 1,95 GH 0.75 240 8 180

Maxcoin DigitalCruncher 12,6 GH ? ? 9 100

https://0x1d00ffff.github.io/0xBTC-Stats/?page=miningcalculator&#

https://whattomine.com/coins/187-dnr-tribus?utf8=%E2%9C%93&hr=1950.0&p=200&fee=0.
0&cost=0.3&hcost=0.0&commit=Calculate

https://whattomine.com/coins/73-max-keccak?utf8=%E2%9C%93&hr=12600&p=200&fee=0.
0&cost=0.3&hcost=0.0&commit=Calculate
Real World income

28.08.2018 - here are the results of running one cards for one week. i can confirm that this
configuration runs really stable ( no crash in that week )

Coin Bitstream Hashrate [V] power[W] $ / month

0xBTC Zetheron 10 GH 0.75 160 140

DNR/SCRIV/BZL DigitalCruncher 1,3 GH 0.75 160 240

Switching 2 Cores
Next Steps
● look at immersion cooling: oilbath = best possible cooling
● look at new bitstreams
● get more cards
● develop solo mining pool ( no averaging over profit, less fee )
○ develop profit switching ( mine one algo but different coins )
○ develop algo switching ( command line reporgramming? )
Immersion Cooling
Theory
you need a dielectric liquid ( similar to oil ). you put the hardware in it. the liquid can take
much more heat than air. you then bring the hot liquid away, cool it, and bring it back
● 2 phase cooling: boiling liquid.
○ if boiling point is e.g. 61°C and you put the board inside the liquid, the liquid (
and the card ) can not get hotter than 61°C ( thermodynamics )
○ you have to put it inside a closed chamber, with cooling pipes ( cold water
flowing through )
○ the boiled liquid will rise to the top of the chamber and meet the pipes: when
they are colder than 61°C, the gas will become liquid again and fall down to
the ground.
○ complicated to realize. the boiling might be bad for the hardware
○ http://multimedia.3m.com/mws/media/644792O/thermal-management-fluids-fo
r-the-electronic-industry.pdf
● 1 phase cooling: the liquid just gets warm.
○ you can use an open bath
○ but you have to thank about how to cool the liquid

1 phase will be good for mining / fpgas. so we use this.

Coolant

which material should be used?

● transformator oil is well known for electronics
○ but it might damage the tiny computer components ( corrosion, chemical
reaction )
● there are some materials which could fit fpga boards
○ http://multimedia.3m.com/mws/media/64888O/fluorinert-electronic-liquid-fc-40
.pdf
○ https://www.engineeredfluids.com/electrocool

FC-40 EC-100 BC-888 better =

boiling point [°C] 165 more

flash point [°C] 294 180 130 more

vapor pressure [Pa] 287 “low” less

density [kg/m³] 1.8 0.81 0.82 not 1(water)

 viscosity [cSt] 2.2 9.58 10 less (for pump)

heat [J/(K kg)] 1100 2203 2203 more

conductivity[W/(m C) 0.065 0.13 0.13 more

price / liter 500 $ 17 $ 9$

The company “Engineered Fluids” has recommended me to use BC-888 also for fpga. this
and the cheap price told me to buy it.
shipping to europe is quite expensive ( 190USD / 20l ), but i had no other choice.

Heat Removal
There are several devices which bring the heat from a liquid to the air. here a comparison:

“Water Chiller”
● Heat Pump: You take e.g. 400W electricity to take 1200W
away from the water ( and bring 1600W to the air )
● with fancy controller to keep temperature at 27.34 °C if
you want
● you bring the coolant to a temperature even lower than air
temperature
● overkill: we don’t need such cold liquid
● consumes too much energy
● 1000 € for 1000W

“Radiator”
● car component (cheap), for water or oil
● no real documentation available
● this passive cooling might be enough
● 100 €
“Air Heater”
● for heating the air in a house
● its a copper pipe connected to
aluminium fins
● their capacity starts from 10kW ( we
don’t need that much - but this tells us
that every Watt we bring in, will go to
the air )
● they have a inbuilt fan
● we bring the coolant temperature just
above air temperature
● they have a datasheet - 500 € for
“10kW”

“Heat Register”(german - “damper register” ? )

● similar to “Air heater”
● same cool datasheets
● it’s just the pipe without a fan
● you connect your own fan
● pro: you can direct the hot air ( use a hose to
bring it out the house )
● 300 € for “10 kW”

data sheets:
the concrete cooling depends on several factors. the data sheets explain the relation
between:
● air flow V
● heat power PT
● air intake temperature Tp1
● air exit temperature Tp2
● liquid intake temperature Tw1
● liquid exit temperature Tw2
● liquid flow Qw
 ● drop in pressure dpw ( your pump needs to be compensate this )

how to read it? If

● the air flow rate is V = 2100 m³/h
● the fpga consumes PT= 1,9kW
● the liquid flow is QW = 82l/h
● the room temperature is Tp1 = 20°C
then
● the liquids hottest temperature is Tw1 = 60°C
● the liquids lowest temperature is Tw2 = 40°C
● the exit air will be Tp2 = 22,5°C

if room temperature is 15° C AND liquid flow increases to 135 l, then you can consume
3,1kW, and have still Tw1 = 60°C

so it is not so easy to read: but i would say: if you keep the 1,9kW, fixed room temperature
and increase flow rate ( my pump: 1800 l/h ), things will get much colder.. so you can even
decrease airflow V and make it silent.

My conclusion on heat removal:

Chiller is waste of energy. radiator too speculative.

My first intent was to bring an air heater outside the building, circulate liquid to fpgas inside
the building. But this has a strong disadvantage in the winter: imagine 25°C room
temperature, -5°C outside temperature. you can now easiliy chill the liquid to Tw1 = 0°C,
Tw2=5°C. so your open box of liquid is 5°C cold. your liquid hose is also not much warmer
on the outside. Nice. But our problem is air moisture: liquid water will form on the surfaces
and drip into your BC-888. This pollution might be bad for the cards.
So i decided to use a simple pipe fan and a heat register: they are inside the building. so the
lowest Temperature Tw2 can be the room temperature Tp1 ( thermodynamics ). Tw1 will
always be higher as Tw2, so no problems with moisture. The hot air will then be brought
outside the building:

Fan
My Fan has V = 1040 m³/h, 345 Pa pressure, 52 dB loud, 108W, 95€
https://www.iventilatoren.de/rohrventilator-dalap-ap-profi-200-mit-schalter-id1105

you always have to look at the pressure curve:

how to to read? you first have to check the air drop of your system: the heater fins have a air
resitance ( sometimes also in the data sheet ): if it is 100 Pascal, and you use “min”, then
airflow will be 650m³/h ( not the advertised value of 830 ! )

air resistance also depends on flowrate, so a real world system will be the crossing of the
system curve with the ventilation curve:

Pump
You have to circulate the liquid from the fpga to the heat register. i chose a pump for brewing
beer ( 95°C allowed, FKM seals )
https://www.polsinelli.it/en/electric-pump-for-beer-novax-20-b-P1041.htm

a pump has a similar curve for pressure drop over flow rate. so our pump with 1500 l/h will
again be slower ( 1000l/h ? )

How many circuits?

There are several options:
● you circulate BC888. you need one pump and one heat register.
○ very effective
○ need more BC888
● you circulate BC888 with an heat exchanger. you have second ( water ) circle, which
circulates from the heat exchanger to the heat register
○ higher temperature for BC888
○ you can choose standard / used components for the water circle
● …
my immersion cooling setup

second setup:
● BitCool BC-888
● VCU1525
● HDPE tank 31l ( for brewing beer ) - will fit like 4 FPGAs
https://www.bauhaus.info/sonstige-gartenhelfer/floraworld-classic-maischefass/p/161
25460
● mortar buckets as backup tank (not safe for hot temperatures)
https://www.bauhaus.info/moertelkuebel-baueimer/probau-moertelkuebel/p/1394034
9
○ cheap but they smell like hell
 ● 1500 lph, 270W Pump ( for brewing beer ) with FKM seals for up to 95°C
https://www.polsinelli.it/en/electric-pump-for-beer-novax-20-b-P1041.htm
○ italian 20mm fluid connector - will fit a ¾ inch hose
○ high volume, but very loud
● Standard PC with 8 GB RAM, SSD, energy-efficient mid-range CPU
https://www.csl-computer.com/shop/product_info.php?products_id=15016&cPath=11
_353
● 850 W Platinum PSU
https://www.caseking.de/kolink-continuum-80-plus-platinum-netzteil-modular-850-wat
t-mit-kaltgeraetekabel-gebu-201.html
● PCIe risers + cables
● 8 kg heat register
https://www.abluft24.de/heizregister/warmwasser-heizregister-fuer-rundrohrsysteme
● high volume fan
https://www.iventilatoren.de/rohrventilator-dalap-ap-profi-200-mit-schalter-id1105
● exhaust air hose
● fuel hose 19mm 27€/m ( safe choice for BC-888 )
https://www.landefeld.de/gruppe/en/tank-truck-suction-pressure-hose-nbr-en-12115/T
WSSP19
● water hose 19mm 4€/m ( cheap choice for BC-888 - PVC might be a problem? )
https://www.landefeld.de/gruppe/de/pvc-schlaeuche-mit-gewebeeinlage-lebensmittel
qualitaet/TX4
● adaptors for connecting 19mm hose to 3/4 ‘’ register, brackets
● vcu backside cooling
○ https://www.reichelt.de/waermeleitfolie-doppelseitig-klebend-100x100mm-wl-f
olie-404-11-p112269.html?r=1
○ good fan:
https://www.reichelt.de/luefter-40x40x20mm-12-v-0-115-a-fan-ml-4020-12-p7
6106.html?r=1
○ cheap fan:
https://www.reichelt.de/luefter-12v-dc-40x40x10mm-u-min-5800-fan-4010-12v
-p11004.html?r=1
○ https://www.reichelt.de/profilkuehlkoerper-100x100x40mm-1-3k-w-v-pr173-10
0-p35401.html?r=1
● steel container ( for food , “Gastronorm 1/1”), of course the version without holes: 53
x 32.5 x 20 cm
https://www.metro.de/cat/de/products/product/440229001001/APS-GN-Beh%C3%A4l
ter-1/1-200-mm-Edelstahl-perforiert

video cooming soon ( first week of september )

first important results:

● its fun to build this. ( but the BC-888 no fun - very oily )
● you get very nervous when something crashes. but then you are happy when
everything runs
● you do need the metal cooling blocks intended for air - the pure chip will go to 100°C
in 5 seconds, even in the fluid
 ● i would leave the stock version of the passive card just as it is - just remove the
backside cover
● cooling the liquid ( heat register ) is absolutely not crucial at the moment - with one
card in 20 liters you just make it a little bit warmer than ambient temperature - so it
might even be okay to leave that part away and just use heat dissipation ( warm fluid
gives heat directly to air over its big surface )
○ the backside of the card ( read with LT box ) is directly cooler than in air
● the liquid stays very cold: like 4°C above room temperature ( when you measure
anywhere in the tank, not directly near the card )
○ you better choose a silent pump: less flow will be ok
○ smaller, silent fan will be ok
● it is very crucial to have a good “fluid flow” - ( like good air flow for fans ). so you take
the output hose of the tank and place it directly on the heat sink of the fpga - then the
chip always sees fresh, cool fluid - it has so possibility of collecting heat and getting
warm
○ bad fluid flow: 80°C in vivado
○ good fluid flow: 50°C in vivado
● small fans: bigger fins is better. maglev is better

0xBTC - 0.74V - 10 GH comparison:

Miner LT Temp_1 LT Temp_2

Airflow bazooka 70°C 76 °C 80 °C

airflow antminer 70 73 76

Fluid without flow 83°C 55°C 75°C

Fluid with flow 48°C 52°C 64°C

0xBTC - overclocking

GH Volt Watt Error Miner Temp_1 Temp_2

10 0,74 190 0% 48°C 52°C 64°C

10 0,72 170 0% 46°C 52°C 62°C

11,5 0,72 210 0% 51°C 52°C 67°C

13 0,72 235 0% 52°C 54°C 71°C

15 0,72 crash
 15 0,75 340 0% 57 °C 57 °C 82 °C

15 0.78 380 60 60 85

15 0,8 390 33 % 60 °C 60 °C 88 °C

17 0,8 380 33 % 58 °C 60 °C 90 °C

With ~ 370 Watt Power consumption we see in LT FAULT_WARN_LIST =

“IOUT_OC_FAULT”, STATUS_IOT = (0x80) IOutOverCurrentFault . this makes totally
sense.. this is just too much current. remember that we have one 8-pin and one 6-pin power
connector. so PCIe power specifications allow us to draw 150W + 75W = 225W !

at the moment this is too much for me.. also Temp_2 gets too high - this is probably because
i flow cool only the front side of the fpga - not the back side.

steel container 0xbtc 0.73V

GH Miner LT Temp_1 LT Temp_2

10 50°C 58 °C 60 °C

11,5 54°C 62°C 66°C

13 55°C 67°C 70°C

15 ( 0.74V ) 70 85 90

15 ( hotter card ) 80 90 95

tribus - overclocking

cores MH Volt Watt Miner Temp_1 Temp_2

1 650 0,75 70 43°C 50°C 55°C

2 1300 0,75 150 52°C 54°C 63°C

3 1950 0,75 245 58°C 56°C 76°C

3 1950 0,73 230 55°C 54°C 73°C

4 “2600” 0,73 230 55°C 54°C 73°C

as we see, there is a maximum of 3 useable cores.

Nexus Algo ( 18.10.2018 )

MH Volt Watt Miner Temp_1 Temp_2

1500 0,86 350 75°C 86°C 94°C

400 0,60

Real World income

02.09.2018

Coin Bitstream Hashrate [V] power[W] $ / month

0xBTC Zetheron 13 GH 0.72 235 200

DNR/SCRIV/BZL DigitalCruncher 1,95 GH 0.73 230 450

Switching 3 Cores

11.09.18
Coin Bitstream Hashrate [V] power[W] $ / month

0xBTC Zetheron 13 GH 0.72 235 200

DNR/SCRIV/BZL DigitalCruncher 1,95 GH 0.73 230 280

Switching 3 Cores

22.09.18
Coin Bitstream Hashrate [V] power[W] $ / month

0xBTC Zetheron 13 GH 0.72 235 300

0xBTC Zetheron 15 GH 0.74 360 350

DNR/SCRIV/BZL DigitalCruncher 1,95 GH 0.73 230 320

Switching 3 Cores
 ZP DigitalCruncher 12 GH 0.73 280 350
21 Cores

next step

i will create smaller tanks - one tank per fpga. it will be a lot eaiser to handle. you can control
the flow of the liquid better. each gets its own small ( quiet ) pump with 13mm flexible hose. i
will pull the fluid from the back of the card ( create a little liquid flow there ) and pump it in the
inside of the card ( big liquid flow here ). the fluid will go through an heat exchanger.

then we have a second circle of fluid - this can just be water with standard components (
pump, heat exchanger ). the circle will go through all the single heat exchangers of the
BC888-circles.