Showing posts with label Data Science. Show all posts
Showing posts with label Data Science. Show all posts

Thursday, 12 April 2018

Data Privacy and New GDPR


What data is included in GDPR

Data covered under GDPR includes any data or information that can be used to identify a person, whether directly or indirectly. This data must be protected under the new regulation.

GDPR - General Data Protection Regulation , EU

Data Protected under GDPR, that can be used to identify an individual includes;
  • Name 
  • Photo 
  • Email address 
  • Banking information 
  • Medical information
  • Computer IP address

Requirements for Companies Under GDPR, 

  •  Companies must prevent the loss, alteration or distribution of customers’ personal and sensitive data. 
  •  Companies will be required to maintain and update a repository that defines the mapping between each item of sensitive data and the corresponding anonymization rule
  • They must also find new ways of managing data collected through mailing lists, online forms, business cards and, in general, marketing and sales activities.
GDPR compliance projects will have a noticeable impact on the information management processes, including: 
  • Data identification and categorization, 
  • Data encryption and masking, 
  • Data supervision, security and data protection, 
  • Compliance audits and reporting.


Readup source:

Tuesday, 11 July 2017

Mathematical Corporation


Book : The Mathematical Corporation

Merging Machine Intelligence (#AI #BigData #ML) with Human Ingenuity for optimised systems, Hence the #MathematicalCorporation.

The leadership must develop  and utilise data and mathematical driven strategies to maintain gain competitive advantage and survive in this time #climate change, increasing demand for limited or depleting resources.

Must Read:

The Mathematical Corporation: Where Machine Intelligence and Human Ingenuity Achieve the Impossible



Thank you @BoozDataScience, I have received my Winners Free copy of The #MathematicalCorporation. Merging Machine Intelligence (#AI #BigData #ML) with Human Ingenuity for Optimised systems, Hence the need #MathematicalCorporation. 

Wednesday, 7 June 2017

Learn About Code Editor




A Programmer's Tool - Learn About Code Editor. 

Code Editor.

What is a Code Editors?


A source code editor is a text editor program designed specifically for editing source code of computer programs by programmers.
It may be a standalone application or it may be built into an integrated development environment (IDE) or web browser.
Source code editors are the most fundamental programming tool, as the fundamental job of programmers is to write and edit source code. (Source code editor)

Source Code Editor versus Text Editor

Source code editor  - enhances, automates and eases the production and editing of source code, it's a more advanced text editor.
Text editor -  are a type of program used for editing plain text files, it provides the very basic functionality for making changes to Config files. Some of the basic features of a text editor may
include; Find and replace, Cut, copy, and paste, Text formatting, Undo and redo, Data transformation, Filtering, Syntax highlighting, Extensibility

Tools to help you code more efficiently

Listed below are some of the best source code editors available to programmers. Some may have steep learning curves over others, but before choosing make sure you understand the features to look of for in a great code editor (scroll down)
  1. Notepad++
  2. Atom
  3. Vim
  4. Light Table
  5. Bluefish
  6. Sublime
  7. Vim
  8. Emacs
  9. Visual Studio Code

Notepad + +

 

Features to look out for in a great code editor

This source code editor features help you work more efficiently while writing code or developing a web application, by enhancing, automating or ease the editing of code.
  • Syntax highlighting for multiple programming and scripting languages
  • Autocompletion
  • Auto Indentation
  • Code folding
  • Text folding
  • Function list
  • Brace matching
  • Support for other programming features - Text shell integration, Graphical shell integration, Macro language, Collaborative editing, Large file support, Long line support, Multi-line regex support, Search in files.
  • Playback and Recordable macros (keyboard or mouse strokes, cache your progress automatically so nothing is lost if you accidentally close the program) to help you code more efficiently
  • History support for calling up commonly used snippets of code
  • Built-in script for automation and customization
  • Flexible customization options and plugins
  • Provide a convenient way to run a compiler, interpreter, debugger, or other programs
  • Syntax  highlight, checking, validation and error notifications
  • Code compression - converting common keywords into single-byte tokens, removing unnecessary whitespace, and converting numbers to a binary form
  • Code uncompression - for viewing, pretty printing, formatting or beautifying code
  • Greater support for specific programming language over others languages (Comparison of text editors - Overview )
  • Support for common document interfaces (i.e. Multiple instances, Single document interface, Single document window splitting, multiple document interface )
  • MDI(multiple document interface): overlapped windows, tabbed document interface, window splitting)
  • Support for basic editing features - Spell checking, Regex-based find & replace, Encoding conversion, Newline conversion, Multiple undo/redo, Rectangular block selection
  • Support for key bindings -  a keyboard shortcut and may be customizable to a user's preference and that program functions may be 'bound' to a different set of keystrokes instead of or in addition to the default
  • Support for remote file editing over network protocols (FTP, HTTP, SSH, WebDAV)
  • Support for some of the most common character encodings (ASCII, ISO-8859,DOS (OEM), EBCDIC,UTF-8, UTF-16)
  • Availability of built-in package managers,  used to search for expansion options from within the program, without the need to fire up your web browser.
  • Instant previews of code with availability of the embedded web browser (i.e Light Table)

Atom

Visual Studio Code Editor


Other Editors

Hex Editors - used for editing binary files. Its also known as binary file editor or byte editor. allows for manipulation of the fundamental binary data that constitutes a computer file The name 'hex' comes from 'hexadecimal': a standard numerical format for representing binary data.
Comparison of hex editors

Stream Editor – used for non-interactive editing

Related Readup





Saturday, 16 May 2015

The Beauty in Data


Beauty in data is not merely in its collection and storage, 
 rather in its processes; 
Preparatory nurturing - error handling, cleaning, missing values, duplications, merging,
aggregation, summarization.
   EDA - analysis and visualization, 
telling stories, 
 giving insights and 
 supporting decision making.
The Data Science Research Center

Thursday, 2 May 2013

Quants: The Alchemists of Wall Street Quants


Quants: The Alchemists of Wall Street

Quants ( also known as Quantitative Analyst) are the math wizards and computer programmers in the engine room of our global financial system who designed the financial products. 
Educational background will include physics, engineering, or mathematics backgrounds rather than economics-related fields, major source of employment for people with physics and mathematics Ph.Ds. Typically, a quantitative analyst will also need extensive skills in computer programming, most commonly C++ and/or Java.

The global financial system has become increasingly dependent on mathematical models trying to quantify human (economic) behavior.  
                  

What Computer Programming Skills would a Quants(Quantitative Analyst) need?:

  • C++
  • Java
  • Python

Tuesday, 30 April 2013

Social Media Data Extraction Algorithms and Stock Exchange Market Effect



The use of algorithms to find patterns in massive amounts of data, executing trading decision based upon the frequency of certain keywords across millions and millions of messages, is becoming more frequent. "The incorporation of social media in these algorithms is relatively new and is therefore particularly susceptible to hacking. '


More Readups:

Twitter Speaks,  Stock Markets Listen and Fears Rise

Could Twitter Crash the Markets? It's Already Causing Mischief.





Anaconda Python Distribution Python 3.3 linking



Anaconda Python Distribution

Anaconda is a pre-packaged Python distribution for scientific users. [1]
Direct your browser to http://continuum.io/downloads.html ; download the version for your machine. Then follow the steps described for your machine here: http://docs.continuum.io/anaconda/install.html. You do not need to worry about setting the paths yet. By default, the Anaconda Python distribution uses Python 2.7 – we want Python 3.3. For this reason, we need to create a corresponding environment. In a shell (see below for opening one), go to the directory where you installed Anaconda.
On Windows, type:
cd Scripts conda create -n py33 python=3.3 anaconda


On MacOS / Linux, type:
cd bin ./conda create -n py33 python=3.3 anaconda Accept the list of things to be installed and wait for a bit. This will install an Anaconda Python environment based on Python 3.3 to the envs/py33 subdirectory of your Anaconda installation.

Data Sciencing

The art of data sciencing(i.e living in and with science) has existed right from time immemorial.... Our continuous and sustainable levity, is and has been dependent on evolution of knowledge, Past on from generation to generation, from species to species as data....in our DNA, books or environment.


"" Sherlock Holmes(movie) ---- Data Data Data How can you build a house without bricks ""