HAMAD BIN KHALIFA UNIVERSITY

COLLEGE OF SCIENCE AND ENGINEERING

Towards Residential Proxies detection: an experimental analysis in

the Android environment

BY

W
NIHAL ABDURAHIMAN
IE
A Thesis Submitted to the Faculty of
EV

College of Science and Engineering

In Partial Fulfillment
PR

of the Requirements

for the Degree of

Master of Science

in

Cybersecurity

April 2021

©Nihal Abdurahiman. All Rights Reserved

 COMMITTEE

The members of the Committee approve the thesis of Nihal Abdurahiman de-

fended on April 05, 2021.

Dr. Roberto Di Pietro

Thesis Supervisor

W
Dr. Gabriele Oligeri
Committee Member
IE
EV

Dr. Spiridon Bakiras

Committee Member
PR

Dr. Yusuf Bicer

Committee Chair

Approved:

Dr. Mounir Hamdi, Dean, College of Science and Engineering

ii
 ABSTRACT

Nowadays, Proxy services are increasingly used by both companies to manage

their Intranets and by private users to increase their online privacy. In addition

to classic services, a new type of VPN is recently emerging, called RESIP. The

idea behind this novel architecture is to exploit residential data connections as exit

nodes of the RESIP network, in order to avoid detection based on the blacklisting

of the IP addresses of data centers.

Luminati is currently the largest RESIP provider globally, with over a million

W
residential IPs under their control. According to their claims, residential devices

join their service voluntarily. To recruit these devices, Luminati provides app
IE
developers with an SDK to be integrated into their applications, offering monetary
EV

rewards in return. On installing applications with an integrated Luminati SDK,

the device joins the Luminati network as a proxy node. The traffic from Luminati

clients will then be relayed through it. Luminati officially requires developers to
PR

mention their SDKs’ presence in terms and conditions of the final application,

to safeguard end users. However, this policy is neither strictly implemented nor

monitored.

The main goal of this thesis is to investigate the feasibility of detecting Luminati’s

SDK, both statically and dynamically, to prevent unauthorized use of end-devices

without the owner’s permission.

iii
 TABLE OF CONTENTS

List of Figures vi

List of Abbreviations vii

W
Acknowledgements IE viii

Dedication ix

1 Introduction 1
EV

1.1 Motivations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

1.2 Contribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
PR

1.3 Thesis Organization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

2 Background and Preliminaries 7

2.1 VPN Networks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

2.2 Residential Proxies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

3 Literature Review 21

3.1 Residential Proxies Classification . . . . . . . . . . . . . . . . . . . . . . 21

3.2 VPN Detection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28

iv
4 Detecting Residential Proxies in the Wild 33

4.1 Design Goals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33

4.2 Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34

4.3 Result and Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40

5 Detecting Residential Proxies on Android Devices 42

5.1 Design Goals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42

5.2 Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43

5.3 Result and Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49

W
6 Conclusion IE 54

Bibliography 57
EV
PR

v
 LIST OF FIGURES

2.1 Remote Access VPN . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

2.2 Site to Site VPN [1] . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

2.3 Basic working of proxy server [2] . . . . . . . . . . . . . . . . . . . . . . 15

W
2.4 Working of a reverse proxy [3] . . . . . . . . . . . . . . . . . . . . . . . . 18

2.5 RESIP architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19

IE
3.1 Experimental setup [4] . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
EV

3.2 Results for each case [5] . . . . . . . . . . . . . . . . . . . . . . . . . . . 27

3.3 Methodology used for detecting content modification [6] . . . . . . . . . 28

PR

4.1 GoLang code to scrap apps from Aptoide . . . . . . . . . . . . . . . . . 36

4.2 GoLang code to download app . . . . . . . . . . . . . . . . . . . . . . . 37

5.1 Experimental setup to capture device information . . . . . . . . . . . . . 43

5.2 Sample data of the csv file . . . . . . . . . . . . . . . . . . . . . . . . . . 49

5.3 Number of packets received for each experiment . . . . . . . . . . . . . . 50

5.4 Number of packets send for each experiment . . . . . . . . . . . . . . . . 51

5.5 Number of bytes received for each experiment . . . . . . . . . . . . . . . 51

5.6 Number of bytes send for each experiment . . . . . . . . . . . . . . . . . 52

5.7 Load average for each experiment . . . . . . . . . . . . . . . . . . . . . . 53

vi
 LIST OF ABBREVIATIONS

APK Android Application Package

SDK Software Development Kit

DEX Dalvik Executable

PCAP

W
Packet Capture File
IE
VPN Virtual Private Network
EV

RESIP Residential IP address

UDP User Datagram Protocol

PR

ADB Android Debug Bridge

DoS Denial of Service

P2P Peer-to-Peer

RSA Rivest–Shamir–Adleman Cryptography

CSV Comma Separated Values

vii
 ACKNOWLEDGEMENTS

First and foremost I am extremely grateful to my supervisor, Prof. Roberto Di.

Pietro and my mentor Mr. Maurantonio Caprolu for their advice, continuous

support and patience shown during my thesis work. Their knowledge and immense

experience helped and encourage me during the times I was stuck for ideas during

the research work. It is their kind help and support that helped me to complete

the thesis on time. They helped to push and motivate myself.

W
IE
EV
PR

viii
 DEDICATION

I dedicate my dissertation work to my family and many friends. A special feeling of

gratitude to my loving parents who has always supported me on all my decisions.

W
IE
EV
PR

ix
Chapter 1

Introduction

W
We live in a time where we are connected to the internet all the time for everything.

All our data from banking information to medical records are stored in the cloud.
IE
On one hand we have large multinational corporations trying to track all our
EV

browsing activities and behaviour to better target personalized advertisements,

for creating better web services, for providing better content recommendations,

etc while on the other hand we have hackers and spammers trying to get access to
PR

our sensitive information, spam their advertisements, spreading fake news among

others.

This is where proxies and VPNs come into the picture. They provide privacy

and confidentiality from corporations, hackers and spammers. But hackers and

spammers themselves use these services to hide their identity before performing

malicious activities. So from one side, these services provide security for normal

consumers from hackers but from the other side they also provide security to

the hacker and spammers from revealing their identity. We can argue that such

services are used heavily for malicious usage than for legitimate ones.

1
 Proxy and VPNs acts as a mediator between a client and a target server. If

a client wants to access a resource hosted on a particular server, the client first

sends the request to the proxy server with desired resource and the server details,

the proxy then forwards this request to the server on behalf of the client. The

server on receiving this request, will send the reply back to the proxy server which

gets forwarded back to the client.

The proxy server has its own IP address, when a request is sent to the server

on behalf of the client, the server typically would not know the presence of a client

as the packets received at the server will show the IP address of the proxy. This

W
provides an extra layer of anonymity to the client, as the server thinks that the

client is the proxy server

IE
Since proxy servers and VPNs are heavily being used for malicious purposes,
EV

many of the websites and services are actively blacklisting proxy servers and VPNs

from accessing their web pages and resources. Web services are investing time and

money in developing better algorithms and detection systems to ban them. Since
PR

majority of the proxy servers used for spamming have IP addresses assigned to

data centers, its quite easy for detection mechanisms to blacklist them by looking

at the IP address pattern.

Spammers and hackers are looking at other solutions to avoid detection and

getting blacklisted by servers. This led to the rise of residential proxies. Residential

IP addresses are provided by the ISP of a country to its residents, which are usually

associated with computers, mobiles phones, tablets and home networks. These IP

addresses being assigned to normal consumers are not blacklisted by any of the

web services. Residential proxy providers give access to such residential IP to route

2
the request from the client to the server, guaranteeing delivery of the packets.

The providers of these Residential IPs control have access to millions of resi-

dential devices to route their client traffic through. They claim the devices joined

their network by choice, [4] suggests that many of the RESIP providers could actu-

ally be using malicious methods to get access to the devices without the knowledge

of the device owner.

1.1 Motivations

W
The number of studies related to RESIP proxies is very limited because it being

a relatively new field but at the same time malicious use of RESIP proxies among
IE
scammers and hackers has been steadily rising for the past few years.

Currently there is only one study on detecting if a device acts as a proxy

EV

by checking the PCAP file [5], there are no studies done on detecting if a app

contains a RESIP proxy provider SDK integrated by checking its source code, nor
PR

app profiling techniques.

Luminati [7] is currently the largest and most popular RESIP proxy provider.

They provide SDKs to app developers to integrate Luminati into applications to

make the device a proxy node, when a user installs an application with Luminati

SDK integrated into it their device will become a RESIP host. They also have

a VPN application called HolaVPN, which provides VPN services to the client

but will also use the client device as a VPN node in return. Through the SDK,

Hola VPN and reselling their service to other providers, Luminati has access to

millions of device to use a VPN node making them the largest RESIP provider in

the world.
3
 Even though Luminati requires its app developers to mention the usage of

their SDK in the application terms and conditions, [4] suggests not all application

might not include it. Many of the users are not aware of their device being used

as a proxy node even if the application mentions its use in its terms as most of

them do not read it and also in many cases the application do not mention the

use of Luminati SDK.

1.2 Contribution

W
This thesis work provides mainly two contribution:

1. Analyzes an APK statically and determines if the application has Luminati

IE
SDK integrated into its code. This tells if a given application uses the device
EV

as an exit node. All the apps hosted in the store Aptoide [8] are downloaded

and the APKs are analyzed.

PR

2. Profiling applications using the Luminati SDK. The resources such as battery

usage, CPU usage, network download and upload stats of an android device

are analyzed and logged. Application using the Luminati SDK is installed

and the resources are analyzed and logged. These two logs are used for

profiling.

1.3 Thesis Organization

The paper is divided into 5 sections:

1. Background and Preliminaries: The section is divided into parts

4
 (a) VPN Networks: Describes in brief the history of VPNs, the way they

work, the major types of VPNs commonly used, and the reasons for

their usage by corporations and normal consumers.

(b) Residential Proxies: Starts with an explanation of what a proxy server

is and how it is different from VPNs, the most common types of proxies

in use, the reasons for their usage. Then goes into details on how

residential proxies work and their architecture. Finally its advantages

and disadvantages are compares with respect to VPNs.

W
2. Related Work: This section is again divided into two parts

(a) Residential Proxy classification: This section describes the related works
IE
done in the field of RESIPs. At the time of writing there is only three
EV

papers on this matter, the design goals, methodology and the results

of all those three papers are presented here.

(b) VPN Detection: The section provides literature reviews on the study
PR

done on detecting tradition VPN systems. Their design goals, method-

ology and the results of those papers are presented here.

3. Detecting Residential Proxies in the Wild: This section describes research

design goals and methodology to detect the usage of Luminati SDK inside an

android application and the associated results. Describes in details on how

the application APKs are downloaded from the app store, how the the static

analysis is performed and results of the app detected to contain Luminati

SDK.

4. Detecting Residential Proxies in android devices: This section describes re-

5
 search design goals and methodology to detect the presence of an Luminati

integrated application in android devices, the associated results are also pre-

sented. Describes in details the resources that are monitored, the compar-

isons between the resource usage with and without Luminati and detected

results.

5. Conclusion: Summarizes the research study. Provides in brief the goals of

the thesis work, the achieved results, the limitation and the potential for

future work opportunities.

W
IE
EV
PR

Reproduced with permission of copyright owner. Further reproduction prohibited without permission.