Mal-Adware Detection for URL Security

Executive Summary

Our project is mainly going to focus on developing a tool which would give an output and it will show whether the URL is secure or not.

Let us try to understand this in a simpler way.

A user will enter a URL in the tool that we have developed. As soon as the URL is entered it will start extracting the HTML code. Now, when this HTML code is extracted it would have the code for the whole website. Now, the HTML code is going to be separated with the help of Document Object Model (DOM). Jsoup is a library we are going to implement to separate the HTML code form other languages.

Apart from Jsoup we have also taken the help of one more library named MessgaeDigest. Here it will filter most of the adverts, frames, images and objects. Here, the first step is achieved of extracting the code. Now the second step of generating the signature of that code would start and that is done with the help of MD5.The MD5 message-digest algorithm is easy and simple to implement and provides a fixed size message digest or “fingerprint” corresponding to a message of variable size. This code is then converted into hashes of fixed length.

Now comes the third step of comparing these hashes with a static database named VirusShare. It is a database that contains nearly 300 to 400 files. Each file consists of hashes of 300 to 400 adwares. So, after comparison it will give an output saying which are the harmful adwares present and whether the site is secure or not. If, it is secure then the user can move ahead and search on the site other it will tell the user not to move ahead.

Now, comes the fourth and the last step. As, the technology is advancing daily there are various adwares that are getting created daily and so there would be various adwares that won’t be present in these files also of Virus Share. So, now it is the duty of the administrator to research about the adware and find out whether the adware is harmful or not and if it is harmful the he has to manually add into the static database of Virus Share. As, we are working on a static database it is the duty of the administrator to add the hashes manually every time. So, that the next time the same virus is found then administrator does not have to add once again in the database.

Table of Contents

Click to expand Table of Contents

1 Introduction

1.1 Purpose

1.2 System Overview

1.2.1 Project Planning

1.2.2 Project Objectives

1.2.3 Project Motivation

1.2.4 Functional and Non-functional requirements:

1.2.6 Implementation strategy

1.2.7 System flow chart

1.2.8 System Description

1.2.9 Assumptions and Constraints

1.3 Glossary

2 Management Overview

2.1 Description of Implementation

2.1.1 Planned Deployment

2.1.2 Installation

2.1.3 Approach

2.2 Project Peers

2.3 Project User

2.4 Major Tasks

2.4.1 Prototype of the Project:

2.4.2 Encrypted Malware Database

2.4.3 Test Cases

2.4.4 Project Reports and PPT

2.5 Implementation Schedule

2.6 Security and Privacy

2.6.1 System Security Features

2.6.2 Security Set Up During Implementation

3 Implementation Support

3.1. Hardware, Software, Facilities, and Materials

3.1.1. Hardware

3.1.2. Software

3.1.3. Facilities

3.1.4. Materials

3.2 Documentation

3.3 Personnel

3.3.1 Staffing Requirements

3.3.2 Training of Implementation Staff

3.4 Outstanding Issues

3.5 Implementation

4 Conclusion

APPENDIX A: Project Implementation Report Approval

APPENDIX B: REFERENCES

APPENDIX C: KEY TERMS

APPENDIX D: System Hardware Inventory

APPENDIX E: System Software Inventory

List of Tables

Table 1 Functional and Nonfunctional Requirements

Table 2 Project Peers

Table 3 Document Submission Details

Table 4 Implementation Schedule

Table 5 Security Features

Table 6 Hardware Used

List of Figures

Figure 1 Project Planning

Figure 2 System Design

Figure 3 System Organization Steps

Figure 4 System Organization Steps 2

Figure 5 Prototype View

Figure 6 Virus Share Database Example

Figure 7 List of Test Cases

Figure 8 Implementation of Jsoup Algorithm

Figure 9 Outcome of Jsoup implementation

Figure 10 EasyList Rule Used in Project

Figure 11 Implementation Code of EasyList

Figure 12 Outcome of EasyList Implementation

Figure 13 Implementation of MD5 Algorithm…1

Figure 14 Implementation of MD5 Algorithm…2

Figure 15 Outcome of MD5 Algorithm

1 Introduction

1.1 Purpose

Malware analysis and detection is an upcoming problem of Cybersecurity due to increase in ad links on screen that propagates user to click on or sometime download malicious payload that leads to severe attacks like ransom ware most in common. Our project is based on this problem to implement malware detection tool that will extract all ad links from a source web page(web scraping) and generates their hashes(fingerprints) in order to compare with them stored hashes(repository) and test their status(whether good or bad one).

Malwares detection is of paramount importance to digital era as more than 16 million malwares have been evolved since 2016/2017.Malware detection can be done through various techniques like static or dynamic analysis, but we implemented static approach using MD5 algorithm due to time constraints. No doubt dynamic analysis is a better solution for obfuscated samples as run time behavior (polymorphic) can be better traced.

Malware codes can be embedded into source page of websites or any software applications. It can be Trojans, worms, backdoors or rootkits. On may4, 2010, a small malware code hit around 16000 host computers that generates DOS attack on server. Due to increase in number of sheer malwares, it is becoming quite important to detect their classifications and families. Attackers basically use reuse of codes of previous malwares to generate new one. Malware detection can be done in two ways: Behavior analysis (study of known patterns and behavior) and static signature based (study of signatures and comparing with repository of stored signatures and adding new signatures manually).

Malware detection helps in protection of same or other systems. It is an implementation technique for detecting malicious behaviors. Now days it is very important method used for preventing zero-day attacks (no prior clues).

Attackers using advertisements as their weapon to plan and execute cyber-attacks on end user systems as through advertisements malicious code will spread immediately and can infect thousands of thousands of computers with in a second. This is main point of consideration in our project. Extraction of ad links(malicious) from web page is quite important as these ads contains malicious content that could lead to severe attacks likeDOS, identity theft.

1.2 System Overview

1.2.1 Project Planning

Project Planning comprises of 4 different phases:

Analysis: It involves analysis of malwares and adwares and techniques used to detect and prevent end systems from Malvertising attacks. It involves research of what platform we needed, how to implement code, how to test tool in real time environment.

Designing: It includes designing of graphical user interface. The main part is to design components of system in order to verify correct flow within the specified system, system flow chart, system specifications.

Implementation: It involves implementing basic functionality with help of language (Java) on android platform.

Testing: It involves testing system against desired test cases.

Figure 1 Project Planning

1.2.2 Project Objectives

1. To develop prototype demonstrating successful detection and classification of at least one type of maladvertising.
2. To design and implement detection of malware based on matching patterns and if one malware found with approximate same pattern, we can easily become aware that these malwares belongs to same family.
3. To implementing signature matching mechanism using hashes of websites generating through message digest algorithms
4. To generate hash signatures for advertisements on website using MD5 Algorithm.
5. To find if generated hash has a match within malware database.
6. To provide required evidences in form of documentations.

1.2.3 Project Motivation

1. Increasing number of malwares: Due to increasing number of malwares and their logically changing patterns, there is dire need to analyse all these patterns and classify them in different forms
2. Increasing number of infected websites: As digital age is growing, so as developers developing new websites with innovative way and that set to be main target for attackers who embed malicious code into that site.
3. Increasing number of infected applications: People of every age like surfing, exploring new web application on android devices as they are open to be downloaded from play store, attackers are getting new floor every second to infect android devices which is further vulnerable to browser hijacking, URL injections.

1.2.4 Functional and Non-functional requirements:

Tools	Description
Libraries: Jsoup, messagedigest	Jsoup acts as parser on html web page and message digest used to generate hash of given ad link
Database	A repository called as Virus Share that contains 300 to 400 hashes in one file is embedded in source programming
Simulation of android platform	Android studio 3.2.1
Number of test cases to be generated	At least 5 for different html tags
MS word, PowerPoint	To report details of project, evidence and implementation

Table 1 Functional and Nonfunctional Requirements

1.2.5 Resources imported

• We use list of regular expression from easy list access to match and compare ad links regular expressions within html source tags
• We imported hash stored files from virushare.com to compare generated hash of extracted ad link.

1.2.6 Implementation strategy

Project implementation is based on static analysis using signature-based approach. So, list of new malware signatures would be updated timely in order to detect new malwares (polymorphic in nature).

Four stages of tool implementation:

1. Entering specified URL

Web URL search box in android emulator. Purpose to enter Web URL is to scarp all links on android emulator

2. Web scraping (extraction of all html elements)

Web scraping using jsoup library. Comparing ad links with regular expressions stored in easy list EasyList acts as primary filter that detects adverts from links extracted including unwanted iframes, images and objects. This list consists of around 3000 regular expression of ad links and is used by ad blockers, ad block plus.

3. Generating hashes of ad links embedded in web page (using MD5 Algorithm)

Message Digest5 (MD5) to generate signature of code: The MD5 message-digest algorithm is easy and simple to implement and provides a fixed size message digest or “fingerprint” corresponding to a message of variable size. Message Digest 5 Invented By Ronalt Rivest In 1991 is a cryptographic Function That Is Used To Verify Integrity Of File. It acts as checksum on dataset. It basically takes random data as input and generates fixed size hash (32 hexadecimal code) as output as shown below

e4d909c290d0fb1ca068ffaddf22cbd0

Hashing is a common method to uniquely identify malwares. Hash basically signifies signatures of malwares (fingerprints). The MD5 algorithms requires following five steps to calculate the message digest or hash value of the message.

MD5 works on the basis of five steps.

1. Append Padding Bits.
2. Append Length.
3. Initialize MD Buffer.
4. Process Message in 16-Word Blocks.
5. Merge the Blocks to Produce Output.

4. Comparing with stored hashes

Comparing hashes with stored hashes and if it matches, displaying its status after testing.

1.2.7 System flow chart

Figure 2 System Design

1.2.8 System Description

This section is an interpretation of what we using to implement basic purpose of tool (detection of mal-ad wares). It comprises of following components:

Android Studio:

Android studio is the Integrated Development Environment used for developing application exclusively for Android platform.

It is a strong editor tool for developing creative User interface and emulators for different versions to test and simulate real time environment without having actual Android devices.

It also has a very useful Gradle plug-in using which you can create application files (apks) with different configurations. Moreover, it makes exporting and uploading of applications on play store easy with a single clicking. Better testing Capabilities and flexible features allows user to run main activities efficiently. It is a very useful tool to run test cases on URL connectors.

Hash Based Signatures

Static Malware Detection Using Hash Based Signatures

It is a method of detecting malwares using Signatures of Web Links and matching them with stored hashes. It is called static because it is matched with already stored hashes and administrator adds new hashes manually into repository in order to update it.

Virus Share

It is a repository of malware samples to provide security researchers, incident responders, forensic analysts, and the morbidly curious access to samples of live malicious code.

It consists of 337 plain text files with stored hashes per each line. Files 0-148 are 4.3 MB in size with 131,072 hashes and 149 onwards are 2.1 MB with 65,536 hashes

Cryptographic Algorithm

MESSAGE DIGEST 5(RONALD RIVEST, 1991): It is an algorithm that works as checksum to verify integrity of valid documents. The basis lies in hashing techniques. It applies hash function on files (URL, DOCUMENTS) and generate their hash as shown below

e4d909c290d0fb1ca068ffaddf22cbd0

1.2.9 Assumptions and Constraints

• Time: Project involves analysis as well as implementation in the prescribed timeline which is less for or project. 12 weeks period for project with submission of progress report every week to subject coordinator
• Demonstrations the working of project through presentations and project reports (2 times).
• Project is assumed to implement at least one type malware, generating hashes and testing them with listed test cases.
• New user to android emulators: Lack of skills to use android emulator .it involves depth searching during starting phase of project (4-6weeks).
• Research: Project involves high level searching of which techniques to be used and what will be the right path to successful execution of tool.

Project Scope:

• This application will support android systems only.
• We are using static analysis technique, using Message Digest.
• We have limited the scope of adware identification with use of EasyList patterns for classifying extracted links as adware or not
• Our database is limited to file from Virus Share Repository.

Project Deliverables

We basically focused on delivery of 3 to 4 evidences to prove the success of our project: –

1. A user-friendly interface: We would develop an interface where all the things would be presented to user in a very simple manner so that it becomes easier for him/her to understand even though if he/she is not from a computer background. We, would display a message saying that the website is not secure, and it has many malwares that may also affect his system and he should not move further.
2. Malware database in form of hashes: We, have taken help of an external database named VirusShare where there are hash signatures of 300 to 400 malwares and that will help us to detect malwares when scanning of the website would be done.
3. Test Cases: We will be working on different test cases throughout our 12 weeks just to make sure that at last we get a final deliverable that is useful and reliable for the end user to use without any errors.
4. Project report and PPT: We will work on different reports and PPT’s and would be still working in the coming 6 weeks as well just to give evidence to our mentors of how are we working and what have we added in our project and how we reached till the final stage. These are actually proof of our working.

1.2.10 System Organization

This section will highlight the overall working of project and dependencies of one module over another. The whole system is organized and planned to work according to implementation strategy as depicted above.

Figure 3 System Organization Steps

Figure 4 System Organization Steps 2

1.3 Glossary

1. URL: Uniform resource locator that is used for web address .it is used to uniquely identify we source on network.
2. Web Browser: Software application for accessing information on world wide web
3. Hash: It is a function denoted as H(M).For two messages M1,M2 we have two hash function (H(M1)and(H(M2)).It is used in cryptographic algorithms like MD5,SHA-1.The longer the hash generated, less probability of collisions.
4. Regular Expressions: List of characters that are used for searching and manipulating text strings. In this project we embed easy list that contains around 3000 stored regular expressions starting with for extracting ad links like &ad_network, &adbannerid, &act_ads, &adname and many more.
5. Web Signatures: These are fingerprints of web page that assures its origin, identity, status.
6. Web Scraping: Web extraction methods for extracting data from websites. It makes use of methods like web crawling (fetching of web page), content scraping, web indexing.
7. Html Parser: module for parsing HTML file. It involves fetching of web page and use of various methods to parse HTML content.
8. Emulators: acts as simulation one software’s to run user created applications in real time environment

2 Management Overview

This chapter describes about how the implementation of malware analysis works. It specifies all the high-level techniques used and also specifies some important aspects of the project.

2.1 Description of Implementation

2.1.1 Planned Deployment

For our project deployment we use Android Studio software and run our website on a Nexus emulator with the help of HTML Commands. We deploy the system with an incremental approach. As we develop a program we test it, then make further changes and finally only after successful completion we move to the next step in deployment. While deploying the application the issues that should be resolved include:

• Software should run in partial configuration.
• Different versions of Android Studio must be available.
• And finally, we should be able to move back to the previous version of the system if there are problems detected with the new system.

2.1.2 Installation

Below are the installation steps of Android Studio.

• Download Java (JDK) Platform and install.
• Download Android Studio from “https://developer.android.com/studio/”.
• Run the “exe” setup file.
• Always keep the default configurations on installation.

2.1.3 Approach

The implementation approach used is the static malware detection analysis method. We use a static database from VirusShare.com which is a repository that provides a database with live malicious codes that researchers, forensic analysts and incident reporters can use. If the malware detected on our system is not in the database, we will manually add them to our database. (VirusShare). Further analysis will be done of the static analysis against strings, libraries that are linked and all functions.

2.2 Project Peers

The below table describes the roles, responsibilities, titles and contact details in case of any queries or any changes that are needed to be implemented on the system.

NAME	ROLE	CONTACT NO
Alex Ng	Project Co-ordinator	+61 3 94793739
Achhardeep Kaur	Design and Implementation	+61 4 69876192
Danielle Rebello	Test case designing and documentation	+61 4 68469917
Gurleen Kaur	Design and Implementation	+61 4 01879603
Sonic Raithatha	Testing and documentation	+61 4 69733252

Table 2 Project Peers

2.3 Project User

Here, we have got those users who are going to use the tool that we have developed. Mainly our project would have 2 major types of users.

1. Administrator

He is a type of user who has got all the types of control in his hands. He, can make changes in the tool that is developed, add or delete things and make the tool updated with the passage of time. It is his responsibility to add hashes everytime in Virus Share and keep the database updated. So, that all the new adwares that are found are shown when a user enters the URL to check whether the site is safe or not. User will get updated results because of the administrator.

2. External User

Here, it will cover all the users that are from computer background and from non-computer background. So, we have developed a tool in such a manner that whenever the results are shown to the user he will understand the result easily and do not need to study anything for understanding.

We feel because of our project they are only the 2 users who will be using our project.

2.4 Major Tasks

In this section we talk about the major tasks or the deliverables of our project that need to be focused on during the overall implementation of our system. We mainly focus on 3-4 evidences that prove the success of our project.

• Prototype of the project.
• Encrypted malware database.
• Test cases.
• Project report and PPT

2.4.1 Prototype of the Project:

A Prototype of the project involves how the front end of this project looks. The prototype view of the project is as shown in Fig. 2.1. When we input the link “http://www.wired.com” and click the “GET WEBSITE” button we will get a list of all the Adwares that are present on that website, and we can differentiate between Mal-adwares and Adwares.

Figure 5 Prototype View

2.4.2 Encrypted Malware Database

An encrypted malware database in the form of MD5 hashes which was extracted from “VirusShare.com” consists of a list of 200-300 files, each having 300-400 malware samples or malicious codes. The below Fig. 2.2 provides you with a screenshot of the list of few of our MD5 hashes malware sample – VirusShare_00100.zip file.

Figure 6 Virus Share Database Example

2.4.3 Test Cases

In order to ensure that our system is running as expected we develop some test cases in comparison with the system requirements and check whether these requirements have been satisfied or not. Table 2.2 shows a list of all our test cases.

Test Case No	Test Input	Test Scenario	Extracted Html Tag	Type	Test Result
1	URL	Check Valid URL	Img loading="lazy" src=“https://assets.adobedtm.com”	IMG SRC	Redirecting to a different source
2	URL	Check Valid URL	Img loading="lazy" src=“https://media.wired.com”	IMG SRC	Valid Source
3	URL	Check Valid URL	Src defer=“https://c.amazon-adsystem.com”	Script SRC	Valid Source
4	URL	Check Valid URL	Src=“https://js-sec.indexww.com”	Script SRC	Valid Source

Figure 7 List of Test Cases

2.4.4 Project Reports and PPT

Table 2.3 gives details about all documentation submitted for evidence during the design and implementation of this system. ‘

Task no	Week no	Submission date	Description
Task 1	Week 3	13th Aug	Project Progress Report 1 – Uploaded via Turnitin
Task 2	Week 5	27th Aug	Project Progress Report 2 – Uploaded via Turnitin
Task 3	Week 6	3rd Sep	Project Design PowerPoint Presentation – Part A – Presentation done during lecture hours and PPT uploaded via Turnitin
		9th Sep	Project Design Report – Part A – Uploaded via Turnitin
Task 4	Week 7	10th Sep	Project Progress Report 3 – Uploaded via Turnitin
Task 5	Week 9	1st Oct	Project Progress Report 4 – Uploaded via Turnitin
Task 6	Week 11	15th Oct	Project Progress Report 5 – Uploaded via Turnitin
Task 7	Week 12	22nd Oct	Project Implementation PowerPoint Presentation – Part B – Presentation done during lecture hours and PPT uploaded via Turnitin
		31st Oct	Project Implementation Report – Part B – Uploaded via Turnitin

Table 3 Document Submission Details

2.5 Implementation Schedule

From week 1-6 we focused on the design of our Mal-adware Detection System. In this section, we will discuss about how the implementation (from week 7) of our system was managed based on all the tasks that were done in the allotted time period. We also focus on major tasks that were to be completed including any milestones that are dependent on the project. Table 2.4 gives a detailed description of the schedule.

Week No	Start Date	End Date	Tasks Completed
Week 6	3rd Sep	9th Sep	Project prototype design.
			Week 7 due Project Progress Report 3 worked on.
Week 7	10th Sep	16th Sep	Extraction of maladwares through a website link.
Week 8	17th Sep	23rd Sep	Implementation ofMD5 hash function program.
			Week 9 due Project Progress Report 4 worked on.
Week 9	1st Oct	7th Oct	Comparing MD5 hashes – VirusShare database, with the links.
			If it does not exist in the database, we will add them manually.
Week 10	8th Oct	14th Oct	Designing test cases with appropriate inputs.
			Comparing which are adware and which are maladware.
			Week 11 Presentation PPT for rehearsal is prepared.
			Week 11 due Project Progress Report 5 worked on.
Week 11	15th Oct	21st Oct	Integration testing performed with each test case developed.
			Minor improvements/changes are implemented and tested once again.
			Final testing results are recorded.
			Week 12 Final Presentation PPT changes are made.
			Working on Final Implementation Report.
Week 12	22nd Oct	28th Oct	Final Presentation and demonstration of the project.
			Working on Final Implementation Report.

Table 4 Implementation Schedule

2.6 Security and Privacy

Most Mal-adware detection systems tend to have security flaws that may affect the performance of the detection system. We need to secure our system from theft or damage of the hardware, software and misdirection of the service that needs to be provided. (Wikipedia). The privacy of the data that is being stored on the system needs to protect. We need to determine what data in the system needs to be protected from third parties and which data can be shared. (Rouse) .

2.6.1 System Security Features

Some of the security policy requirements that can be implemented on the system to provide better security features can be as follows,

• Network security perimeter points should be controlled to limit the entering of malware without interfering with the appropriate use of the network. Also, it is better to use only firewalls that are IT-approved.
• Personal devices should be used only for business purposes, if and only if the device is authorized by BYOD scheme. Mobile Device Management (MDM) should always be installed and configured, also corporate antivirus software must be used and maintained.
• Limit the spread and infection within the organization and prevent malware as far as possible from leaving the organization.
• Data backups must be done regularly and be retained for up to at least 3 months.
• Email gateways should automatically scan all the emails that traverse through both the inbound and outbound gateways. The automatic scanning should be done through an IT-approved antivirus software used for emails.
• All ICT devices should be monitored, configured, maintained and patched to minimize vulnerabilities.
• Attachments coming through email with non-standard file extensions should be blocked out or stripped at the email gateways.
• Antivirus that are approved should be run continuously wherever applicable. It should automatically scan, fix and isolate malwares.
• Malware signature files should be update as often as possible.
• Always a strong incident management and business continuity plan should be maintained.
• Proper training should be given to all employees before using the system, so that they will know the risk of mal-adware attacks. (Malware – Malicious software)

Item	Relevance
Policy on: cybersecurity; IT system security; monitoring and surveillance; integrity; access control	Security controls at the perimeter and our networks internally need to be protected against malware injection.
Policy on: BYOD	Specifies controls to the security on personal ICT devices used at work.
Policy on: Incident management and reporting	Malware incidents should be reported at the helpdesk immediately. Incidents will be resolved promptly by experts.
Policy on: Business Continuity	Mal-adware related events are common and serious enough to document the recovery steps, as well as contingency plan.
Policy on: Backups and Archives	Securely retaining offline copies of all important files that you will need to recover from malwares, ransomware attacks and other incidents.
Policy on: Antivirus controls	Provides details about the antivirus and other control measures.

Table 5 Security Features

2.6.2 Security Set Up During Implementation

When we talk about the security issues we don’t feel our project will face any serious security related issues but still we will create an account with user ID and password so that the user can login and carry out the activities whenever he wants to and test for different websites and check whether the URL is secure or not.

3 Implementation Support

3.1. Hardware, Software, Facilities, and Materials

As our team worked on Android studio for the implementation of project, So, we did not require too much of the hardware equipment. All the components that we used during implementation of this project are listed in section.

3.1.1. Hardware

Following is the list of hardware requirements to develop our project using Android Studio:

Hardware	Minimum Requirement	Used by Our team
Operating System	64-bit distribution capable of running 32-bit applications	64- bit Windows 10
RAM	3 GB minimum	8GB + 1 GB for the Android Emulator
Hard Drive	2 GB of available disk space minimum	4 GB (500 MB for IDE + 1.5 GB for Android SDK and emulator system image)
Screen Resolution	1280 x 800 minimum screen resolution	1280 x 800 minimum screen resolution
Processor	Intel® processor with support for Intel® VT-x   Intel® EM64T (Intel® 64), and Execute Disable (XD) Bit functionality AMD processor with support for AMD Virtualization™ (AMD-V™)	Intel® processor with support for Intel® VT-x

Table 6 Hardware Used

All this information has also been recorded in Hardware Inventory table in Appendix D.

3.1.2. Software

Android Studio is the official integrated development environment for Google’s Android operating system, built on JetBrains’ IntelliJ IDEA software and designed specifically for Android development. It is available for download on Windows, macOS and Linux based operating systems.

Android Studio provides the fastest tools for building apps on every type of Android device. We have used latest version of Android Studio 3.2.1 for Windows 64-bit. The default build tools version is now 28.0.3.

Other tools used for development of program:

Development of graphical user interface to scan and show status of the website

Tool used: –

• HTML
• CSS

Here, the system must have GUI where user can upload URL or file for malware scanning

Approach to detect malware with a Hash Signature

Tool used: –

• MD5 algorithm
• The complete implementation is done with Java

Management of the malware

Tool used: – Web browser

Message will be displayed on screen that source is not secure and will also

show how many errors are there.

Refer to the Software Inventory table in Appendix E.

3.1.3. Facilities

Our project implementation does not require hard the physical facilities But yes, we had spent total of 12 weeks for development of this project which consists of 7 hours per day. We already mentioned all the anticipated dates of intermediate document submissions in Section 2.3, 2.4. We submitted everything on time and we were successful to satisfy documentation requirements of the project.

3.1.4. Materials

Virus Share: VirusShare.com is a repository of malware samples to provide security researchers, incident responders, forensic analysts, and the morbidly curious access to samples of live malicious code.

Access to the site is granted via invitation only. The use of bots, scripts, or other methods to scrape data from the site, download samples at an excessive rate, or otherwise affect general performance of the site or backend systems will not be tolerated. VirusShare is a free site.

This site contains malware. VirusShare.com is not responsible for any damage, infection, breach or other incident that may result from accessing this website and displaying or downloading any information.

EasyList: EasyList is the primary filter list that removes most adverts from international web pages, including unwanted frames, images and objects. It is the most popular list used by many ad blockers and forms the basis of over a dozen combination and supplementary filter lists.

We have used latest EasyList details are as following:

Version of Easy List: 201810301450

Last modified Timestamp: 20 Oct 2018 14:50 UTC

Homepage address: https://easylist.to/

License details may be seen at: https://easylist.to/pages/licence.html

3.2 Documentation

This Project Implementation Report is the only document to be used as final evidence of this project. We haven’t used any additional documentation to support the deliverable system.

3.3 Personnel

This project implementation has been done by four team members. And we completed full project in total span of 12 weeks, that includes requirement analysis, design, implementation and documentation.

3.3.1 Staffing Requirements

Required Skills:

Expertise in one of programming language we used java.

Knowledge of Software Development Life cycle.

Experience of writing professional documents.

3.3.2 Training of Implementation Staff

• System support: Training was providing during week 3 to week 8 on different aspects of Project Management and System Design
• System maintenance and modification: Training has been provided by project supervisor om System Development Lifecycle to understand how to maintain system while process of development and GIT & GitHub tools those can be used for working in team to share partial versions of project. Training was provided in Week 2 on 6th August 2018.

3.4 Outstanding Issues

• Difficulties in deciding platform to work on.
• We were not able to estimate time required to implement the technique we have chosen.
• Learning new technology was one of biggest issues.
• As all team members are working and having different another subject, so we were facing some issues to meet for project meetings and work together.

3.5 Implementation

Jsoup library as HTML Parser

jsoup is a Java library for working effectively with HTML in real world. jsoup can parse Hypertext markup language files, URLs with html code, strings or input streams. It makes extraction of data from HTML easy by offering traversal through Document Object Model (DOM) methods, jQuery selectors and Cross Site Scripting (jsoup, 2009-2018).

It implements the WHATWG HTML5 specification and divide HTML code into different section based on tags.  Features of jsoup library as modern browsers are as following:

• We used it to extract HTML code from a URL or HTML files.
• It was used finds and separates data,  using  CSS  selectors  Document  Object  Model  (DOM) traversal
• To prevent XSS attacks, it clear user submitted content from forms in against of a safe whitelist, jsoup is intended to work with all kinds of HTML elements found on webpage.

Screenshots

Implementation of Jsoup to parse webpage:

Where

•     url– uniform resource locator of the html page to be loaded.
•     document− document is an object which represents the HTML Document Object Model
•     connect(url)−This function/method makes a new Connection with specified url or file,
• get()−It is a function to fetch and parse a HTML from url.
• Jsoup− main class library to connect the specified url.

Figure 8 Implementation of Jsoup Algorithm

Outcome of using Jsoup Library:

Figure 9 Outcome of Jsoup implementation

Implementation of EasyList: It is the primary filter list that removes most adverts from international web pages, including unwanted frames, images and objects. It is the most popular list used by many ad blockers and forms the basis of over a dozen combination and supplementary filter lists.

Easy list uses some Simple blocking rules

1. Blocking by address parts
2. Blocking by domain name
3. Blocking exact address

We have used rule no. 1 which is blocking by address parts. This rule defines which addresses it is to be blocked or detected.

Figure 10 EasyList Rule Used in Project

Figure 11 Implementation Code of EasyList

Outcome of EasyList: Identification of adwares from all links extracted.

Figure 12 Outcome of EasyList Implementation

Implementation of Message Digest Code Algorithm (using java.security.MessageDigest)

This MessageDigest class delivers the functionality of a message digest algorithm, for example, MD5 or SHA. These digests are one-way hash functions. This MessageDigest class delivers the functionality of a message digest algorithm, for example, MD5 or SHA. These digests are one-way hash functions. These hash functions take data of random size and gives back a hash value of fixed-length (Rivest, 1992).

To create a new Java MessageDigest instance corresponding to hash function we have to call the getInstance() function from class MessageDigest as following:

MessageDigest messageDigest = MessageDigest.getInstance(“MD5”);

Screenshot of code:

Figure 13 Implementation of MD5 Algorithm…1

Figure 14 Implementation of MD5 Algorithm…2

Screenshot of outcome:

Figure 15 Outcome of MD5 Algorithm

java.io.IOException

If an error occurs whilst fetching the URL, it will throw an IOException, which you should handle appropriately. We have to handle an IOException either by adding throws IOException or with a try-catch block

4 Conclusion

So, this project mainly focuses on developing a tool and telling whether the URL is secure or not. Generally, we have tried to develop our project in such a way that the steps as well as the generated results are understood to someone who is from a non-Computer and I.T. background and he too can understand what the generated result says when he is shown the output. Here, we have tried to work on this project in such a way that we have kept in mind people from all the backgrounds.

APPENDICES

APPENDIX A: Project Implementation Report Approval

The undersigned acknowledge that they have reviewed the Implementation Report and agree with the information presented within this document. Changes to this Project Implementation Report will be coordinated with, and approved by, the undersigned, or their designated representatives.

Signature:		Date:
Print Name:
Title:
Role:	Project Manager

APPENDIX B: REFERENCES

The following table summarizes the documents referenced in this document.

Document Name	Description	Location
advertisement (malicious advertisement or Malvertising)	Definition of Malvertising	https://searchsecurity.techtarget.com/definition/malvertisement-malicious-advertisement-or-malvertising
Python Central	Description of Python	https://www.pythoncentral.io/category/python-how-tos/
The Conversation	How malwares gets inside your apps	http://theconversation.com/explainer-how-malware-gets-inside-your-apps-79485
What is MD5	Description of MD5	https://www.lifewire.com/what-is-md5-2625937
W3Schools.com	HTML Attributes	https://www.w3schools.com/html/html_attributes.asp
Trend Micro	Malvertising: when online adds attack	https://www.trendmicro.com/vinfo/us/security/news/cybercrime-and-digital-threats/malvertising-when-online-ads-attack
Talent Cookie	Static Malware analysis(Stepwise Guide)	https://www.talentcookie.com/2016/11/static-malware-analysis-lets-start-with-some-basics/
Iframe Hacking	What is Iframe Hacking?	http://www.how2lab.com/internet/security/iframe-hacking.php
Bank Info Security	Why Malvertising attacks won’t stop	https://www.bankinfosecurity.com/malvertising-attacks-slam-yahoo-a-8451
Tech Target	Data Privacy	https://searchcio.techtarget.com/definition/data-privacy-information-privacy
Virus Share.com	Definition of Virus Share	https://virusshare.com/about.4n6

APPENDIX C: KEY TERMS

The following table provides definitions and explanations for terms and acronyms relevant to the content presented within this document.

Term	Definition
Malware	Software that is specifically designed to damage and gain illegal access to the computer.
Jsoup	Jsoup is an open source library that is designed to parse, extract and change the data that is stored in HTML documents.
EasyList	EasyList is the primary filter list that removes most adverts from international webpages, including unwanted frames, images and objects
MD5	MD5 is a cryptographic hash function whose main purpose is to verify that the file has been unaltered.
Virus Share	A database where hashes of the adwares are stored in files.
Adware	Software that automatically displays advertising material when the user is online.

APPENDIX D: System Hardware Inventory

Hardware	Usage
Operating System	64-bit Windows 10
RAM	3 GB space
Hard Drive	2 GB available space
Screen Resolution	1280 * 80 is the minimum screen resolution
Processor	Intel® processor with support for Intel® VT-x Intel® EM64T (Intel® 64), and Execute Disable (XD) Bit functionality AMD processor with support for AMD Virtualization™ (AMD-V™)

APPENDIX E: System Software Inventory

Software	Usage
Android Studio	Android Studio is the official integrated development environment for Google’s Android operating system, built on JetBrains’ IntelliJ IDEA software and designed specifically for Android development. It is available for download on Windows, macOS and Linux based operating systems.   Android Studio provides the fastest tools for building apps on every type of Android device. We have used latest version of Android Studio 3.2.1 for Windows 64-bit
Virus Share	A database file that has got hashes of around 300 to 400 adwares

APPENDIX F: Application Software Inventory

Name/ ID

Type

Module/ Version

Developed By Person

App Owner   (Person or Dept)

Maintained By: (Contract? Person or Dept)

Maintenance Contact Point

Maintenance Type/ Level of Coverage

Maintenance Period Expiration Date

Required Approval