.chapter6<-function(i=0){ " i Chapter 6: Open data for finance i Explanations - ---------------------------------- -- ------------------------------------ 1 What does it mean Open data? 21 Census data 2 data packages: search() 22 Big data make simple (data sets) 3 : datasets 23 AWS Public Datasets 4 : data('Titanic') 24 Bureau of Economic Analysis 5 : data('iris') 25 Wall Street Journal, Quandl 6 : sleeping data 26 Option data and free TORQ database 7 : help(data) 27 NYSE daily TAQ sample data 8 all packages: 28 NYSE Bond data (high-frequency data) sample data 9 : data sets 29 2600+ Open Data Portals Around the World 10 : list 30 Four ways to get data 11 .searchDataInPackages() 31 Fix issues with the R quantmod 12 data set from a specific package 32 Bureau of Labor Statistics 13 Yahoo!Finance and Google Finance 33 Quandl 14 French Data Library 34 our .getBSannual() function 15 FRED (Feds Federal Reserve) 35 our .getISannual() function 16 SEC filings data (1994- to today) 36 our .getCFannual() function 17 Machine Learning (UCI) 37 our R data sets 18 Data Science Central 38 our CSV data sets 19 Videos 39 our Excel files 20 Links 40 our text files Example #1:> .c6 # see the above list Example #2:> .c6(1) # see the first explanation ";.zchapter6(i)} .n6chapter<-40 .zchapter6<-function(i){ if(i==0){ print(.c6) }else{ .printEachQ(6,i,.n6chapter) } } .c6<-.chapter6 .C6EXPLAIN1<-" ////////////////////////// ////////////////////////// " .C6EXPLAIN1<-"What does it mean 'open data'? /////////////////////////////// Open data means the data publicly available for economics, finance and accounting For teaching purpose, it is more than enough to use open data This is one of three concepts for the Open-Source-Finance Kane, David, 2006, Open-Source Finance, Harvard University, http://papers.ssrn.com/sol3/papers.cfm?abstract_id=966354 ///////////////////////////////// " .C6EXPLAIN2<-" /////////////////////////////// To find the preloaded packages, we can use the search() function. > search() [1] \".GlobalEnv\" \"package:httr\" \"package:stats\" [4] \"package:graphics\" \"package:grDevices\" \"package:utils\" [7] \"package:datasets\" \"package:methods\" \"Autoloads\" [10] \"package:base\" > ///////////////////////////////// " .C6EXPLAIN3<-"data sets inside the package 'datasets' /////////////////////////////// data(packag='datasets') Data sets in package ‘datasets’: AirPassengers Monthly Airline Passenger Numbers 1949-1960 BJsales Sales Data with Leading Indicator BJsales.lead (BJsales) Sales Data with Leading Indicator BOD Biochemical Oxygen Demand CO2 Carbon Dioxide Uptake in Grass Plants ChickWeight Weight versus age of chicks on different diets DNase Elisa assay of DNase EuStockMarkets Daily Closing Prices of Major European Stock Indices, 1991-1998 Formaldehyde Determination of Formaldehyde HairEyeColor Hair and Eye Color of Statistics Students Harman23.cor Harman Example 2.3 Harman74.cor Harman Example 7.4 Indometh Pharmacokinetics of Indomethacin InsectSprays Effectiveness of Insect Sprays JohnsonJohnson Quarterly Earnings per Johnson & Johnson Share LakeHuron Level of Lake Huron 1875-1972 LifeCycleSavings Intercountry Life-Cycle Savings Data Loblolly Growth of Loblolly pine trees Nile Flow of the River Nile Orange Growth of Orange Trees OrchardSprays Potency of Orchard Sprays PlantGrowth Results from an Experiment on Plant Growth Puromycin Reaction Velocity of an Enzymatic Reaction Seatbelts Road Casualties in Great Britain 1969-84 Theoph Pharmacokinetics of Theophylline Titanic Survival of passengers on the Titanic ToothGrowth The Effect of Vitamin C on Tooth Growth in Guinea Pigs UCBAdmissions Student Admissions at UC Berkeley UKDriverDeaths Road Casualties in Great Britain 1969-84 UKgas UK Quarterly Gas Consumption USAccDeaths Accidental Deaths in the US 1973-1978 USArrests Violent Crime Rates by US State USJudgeRatings Lawyers' Ratings of State Judges in the US Superior Court USPersonalExpenditure Personal Expenditure Data UScitiesD Distances Between European Cities and Between US Cities VADeaths Death Rates in Virginia (1940) WWWusage Internet Usage per Minute WorldPhones The World's Telephones ability.cov Ability and Intelligence Tests airmiles Passenger Miles on Commercial US Airlines, 1937-1960 airquality New York Air Quality Measurements anscombe Anscombe's Quartet of 'Identical' Simple Linear Regressions attenu The Joyner-Boore Attenuation Data attitude The Chatterjee-Price Attitude Data austres Quarterly Time Series of the Number of Australian Residents beaver1 (beavers) Body Temperature Series of Two Beavers beaver2 (beavers) Body Temperature Series of Two Beavers cars Speed and Stopping Distances of Cars chickwts Chicken Weights by Feed Type co2 Mauna Loa Atmospheric CO2 Concentration crimtab Student's 3000 Criminals Data discoveries Yearly Numbers of Important Discoveries esoph Smoking, Alcohol and (O)esophageal Cancer euro Conversion Rates of Euro Currencies euro.cross (euro) Conversion Rates of Euro Currencies eurodist Distances Between European Cities and Between US Cities faithful Old Faithful Geyser Data fdeaths (UKLungDeaths) Monthly Deaths from Lung Diseases in the UK freeny Freeny's Revenue Data freeny.x (freeny) Freeny's Revenue Data freeny.y (freeny) Freeny's Revenue Data infert Infertility after Spontaneous and Induced Abortion iris Edgar Anderson's Iris Data iris3 Edgar Anderson's Iris Data islands Areas of the World's Major Landmasses ldeaths (UKLungDeaths) Monthly Deaths from Lung Diseases in the UK lh Luteinizing Hormone in Blood Samples longley Longley's Economic Regression Data lynx Annual Canadian Lynx trappings 1821-1934 mdeaths (UKLungDeaths) Monthly Deaths from Lung Diseases in the UK morley Michelson Speed of Light Data mtcars Motor Trend Car Road Tests nhtemp Average Yearly Temperatures in New Haven nottem Average Monthly Temperatures at Nottingham, 1920-1939 npk Classical N, P, K Factorial Experiment occupationalStatus Occupational Status of Fathers and their Sons precip Annual Precipitation in US Cities presidents Quarterly Approval Ratings of US Presidents pressure Vapor Pressure of Mercury as a Function of Temperature quakes Locations of Earthquakes off Fiji randu Random Numbers from Congruential Generator RANDU rivers Lengths of Major North American Rivers rock Measurements on Petroleum Rock Samples sleep Student's Sleep Data stack.loss (stackloss) Brownlee's Stack Loss Plant Data stack.x (stackloss) Brownlee's Stack Loss Plant Data stackloss Brownlee's Stack Loss Plant Data state.abb (state) US State Facts and Figures state.area (state) US State Facts and Figures state.center (state) US State Facts and Figures state.division (state) US State Facts and Figures state.name (state) US State Facts and Figures state.region (state) US State Facts and Figures state.x77 (state) US State Facts and Figures sunspot.month Monthly Sunspot Data, from 1749 to \"Present\" sunspot.year Yearly Sunspot Data, 1700-1988 sunspots Monthly Sunspot Numbers, 1749-1983 swiss Swiss Fertility and Socioeconomic Indicators (1888) Data treering Yearly Treering Data, -6000-1979 trees Diameter, Height and Volume for Black Cherry Trees uspop Populations Recorded by the US Census volcano Topographic Information on Auckland's Maunga Whau Volcano warpbreaks The Number of Breaks in Yarn during Weaving women Average Heights and Weights for American Women ///////////////////////////////// " .C6EXPLAIN4<-"data set: Titanic /////////////////////////////// > data('Titanic') > x<-Titanic > df<-data.frame(x) > dim(df) [1] 32 5 > head(df) Class Sex Age Survived Freq 1 1st Male Child No 0 2 2nd Male Child No 0 3 3rd Male Child No 35 4 Crew Male Child No 0 5 1st Female Child No 0 6 2nd Female Child No 0 ///////////////////////////////// " .C6EXPLAIN5<-"data: iris /////////////////////////////// > data('iris') > x<-iris > df<-data.frame(x) > dim(df) [1] 150 5 > head(df,4) Sepal.Length Sepal.Width Petal.Length Petal.Width Species 1 5.1 3.5 1.4 0.2 setosa 2 4.9 3.0 1.4 0.2 setosa 3 4.7 3.2 1.3 0.2 setosa 4 4.6 3.1 1.5 0.2 setosa > tail(df,4) Sepal.Length Sepal.Width Petal.Length Petal.Width Species 147 6.3 2.5 5.0 1.9 virginica 148 6.5 3.0 5.2 2.0 virginica 149 6.2 3.4 5.4 2.3 virginica 150 5.9 3.0 5.1 1.8 virginica ///////////////////////////////// " .C6EXPLAIN6<-"Sleeping data /////////////////////////////// > data(sleep) > sleep extra group ID 1 0.7 1 1 2 -1.6 1 2 3 -0.2 1 3 4 -1.2 1 4 5 -0.1 1 5 6 3.4 1 6 7 3.7 1 7 8 0.8 1 8 9 0.0 1 9 10 2.0 1 10 11 1.9 2 1 12 0.8 2 2 13 1.1 2 3 14 0.1 2 4 15 -0.1 2 5 16 4.4 2 6 17 5.5 2 7 18 1.6 2 8 19 4.6 2 9 20 3.4 2 10 ///////////////////////////////// " .C6EXPLAIN7<-"help(data) /////////////////////////////// data {utils} R Documentation Data Sets Description Loads specified data sets, or list the available data sets. Usage data(..., list = character(), package = NULL, lib.loc = NULL, verbose = getOption(\"verbose\"), envir = .GlobalEnv, overwrite = TRUE) ///////////////////////////////// " .C6EXPLAIN8<-"list of all packages /////////////////////////////// a<-.packages(all.available = TRUE) > length(a) [1] 187 ///////////////////////////////// " .C6EXPLAIN9<-"List all data sets /////////////////////////////// a<-data(package = .packages(all.available = TRUE)) df<-data.frame(unlist(a)) > dim(df) [1] 2477 1 > head(df) unlist.a. title Data sets results1 datasets results2 datasets results3 datasets results4 datasets results5 datasets > ??? ///////////////////////////////// " .C6EXPLAIN10<-"R data set for all data sets ///////////////////////////////// R data set for all the data sets included in various R packages a<-load(url('http://datayyy.com/fmr2/allDataInPackages.RData')) > a [1] \"df\" > head(df) Package Name Explanation 1 datasets AirPassengers Monthly Airline Passenger Numbers 1949-1960 2 datasets BJsales Sales Data with Leading Indicator 3 datasets BJsales.lead (BJsales) Sales Data with Leading Indicator 4 datasets BOD Biochemical Oxygen Demand 5 datasets CO2 Carbon Dioxide Uptake in Grass Plants 6 datasets ChickWeight Weight versus age of chicks on different diets > tail(df) Package Name Explanation 614 survival udca Data from a trial of usrodeoxycholic acid 615 survival udca1 (udca) Data from a trial of usrodeoxycholic acid 616 survival udca2 (udca) Data from a trial of usrodeoxycholic acid 617 survival uspop2 (survexp) Projected US Population 618 survival valveSeat (reliability) datasets 619 survival veteran (cancer) datasets > ///////////////////////////////// " .C6EXPLAIN11<-".searchDataInPackages() ///////////////////////////////// > .searchDataInPackages('titanic') Package Name Explanation datasets Titanic Survival of passengers on the Titanic rpart.plot ptitanic Titanic data with passenger names and other details removed. ///////////////////////////////// " .C6EXPLAIN12<-"data('aids') /////////////////////////////// > data(\"aids\") Warning message: In data(\"aids\") : data set 'aids' not found > data(\"aids\",package=\"boot\") > x<-aids > dim(x) [1] 570 6 > head(x) year quarter delay dud time y 1 1983 3 0 0 1 2 2 1983 3 2 0 1 6 3 1983 3 5 0 1 0 4 1983 3 8 0 1 1 5 1983 3 11 0 1 1 6 1983 3 14 0 1 0 > ///////////////////////////////// " .C6EXPLAIN13<-"Yahoo!finance and Google finance /////////////////////////////// Yahoo!Finance: We could download data manually Example 1: download historical daily stock price data Step 1: go to http://finance.yahoo.com Step 2: entre a ticker, such as ibm Step 3: click \"Historical Prices\" Step 4: click \"Download \" Example #2: Current term structure of interest rate http://finance.yahoo.com/bonds Google Finance: http://google.com/finance ///////////////////////////////// " .C6EXPLAIN14<-"Prof. French Data Library /////////////////////////////// http://mba.tuck.dartmouth.edu/pages/faculty/ken.french/data_library.html ///////////////////////////////// " .C6EXPLAIN15<-"FRED (Feds Federal Reserve Economic Data Library) /////////////////////////////// FRED (Feds Federal Reserve Economic Data Library) ----------------------------------------------- https://research.stlouisfed.org/fred2/ ///////////////////////////////// " .C6EXPLAIN16<-"SEC filings data (1994- to today) /////////////////////////////// SEC Filings. Company SEC filings represent the financial reports and statements filed with the Securities and Exchange Commission by the company. https://www.sec.gov/edgar/searchedgar/companysearch.html ///////////////////////////////// " .C6EXPLAIN17<-"Free data sets for Machine Learning (UCI) /////////////////////////////// http://archive.ics.uci.edu/ml/ ///////////////////////////////// " .C6EXPLAIN18<-"Data Science Central /////////////////////////////// http://www.datasciencecentral.com/profiles/blogs/big-data-sets-available-for-free ///////////////////////////////// " .C6EXPLAIN19<-"Videos /////////////////////////////// Milken Institute,2019,Open Data, Open Banking: Creating a More Competitive Financial Services Ecosystem (v2k,s38k,t1:04:24) https://www.youtube.com/watch?v=eh_JJgfi0E0 Simpleshow foundationOpen,2016, Data - explained in a nutshell (v12k,s6k,t2:03) https://www.youtube.com/watch?v=c42QNa-rccw TEDx Talks,2014, Open data changes lives | Jeanne Holm | TEDxUCLA (v8k,s21m,t17:47) https://www.youtube.com/watch?v=ThM6umznsWM U of G Library,2018, What is Open Data? (v1k,s1k,t2:49) https://www.youtube.com/watch?v=frPFDAvc15Q ///////////////////////////////// " .C6EXPLAIN20<-"Links /////////////////////////////// Assets Macro: https://www.assetmacro.com/ Kaggle data sets: https://www.kaggle.com/datasets http://usgovxml.com http://aws.amazon.com/datasets http://databib.org http://datacite.org http://figshare.com http://linkeddata.org http://reddit.com/r/datasets http://thewebminer.com/ http://thedatahub.org alias http://ckan.net http://quandl.com http://enigma.io http://www.ufindthem.com/ Interactive Network Data Repository http://NetworkRepository.com http://MLvis.com Open Data Inception http://data.opendatasoft.com http://archive.ics.uci.edu/ml/ http://crawdad.org/ http://data.austintexas.gov http://data.cityofchicago.org http://data.govloop.com http://data.gov.uk/ data.gov.in http://data.medicare.gov http://data.seattle.gov http://data.sfgov.org http://data.sunlightlabs.com https://datamarket.azure.com/ http://developer.yahoo.com/geo/g... http://econ.worldbank.org/datasets http://en.wikipedia.org/wiki/Wik... http://factfinder.census.gov/ser. http://ftp.ncbi.nih.gov/ http://gettingpastgo.socrata.com http://googleresearch.blogspot.c... http://books.google.com/ngrams/ http://medihal.archives-ouvertes.fr http://public.resource.org/ http://rechercheisidore.fr http://snap.stanford.edu/data/in.. http://timetric.com/public-data/ https://wist.echo.nasa.gov/~wist... http://www2.jpl.nasa.gov/srtm http://www.archives.gov/research... http://www.bls.gov/ http://www.crunchbase.com/ http://www.dartmouthatlas.org/ http://www.data.gov/ http://www.datakc.org http://dbpedia.org http://www.delicious.com/jbaldwi... http://www.faa.gov/data_research/ http://www.factual.com/ http://research.stlouisfed.org/f... http://www.freebase.com/ http://www.google.com/publicdata... http://www.guardian.co.uk/news/d... http://www.infochimps.com http://www.kaggle.com/ http://build.kiva.org/ http://www.nationalarchives.gov.... http://www.nyc.gov/html/datamine... http://www.ordnancesurvey.co.uk/... http://www.philwhln.com/how-to-g... http://www.imdb.com/interfaces http://imat-relpred.yandex.ru/en... http://www.dados.gov.pt/pt/catal... http://knoema.com http://daten.berlin.de/ http://www.qunb.com http://databib.org/ http://datacite.org/ http://data.reegle.info/ http://data.wien.gv.at/ http://data.gov.bc.ca interaction data in learning environments) https://pslcdatashop.web.cmu.edu/ http://www.icpsr.umich.edu/icpsrweb/CPES/ http://www.dati.gov.it http://dati.trentino.it http://www.databagg.com/ http://networkrepository.com ///////////////////////////////// " .C6EXPLAIN21<-"Census data /////////////////////////////// http://www.census.gov/compendia/statab/hist_stats.html ///////////////////////////////// " .C6EXPLAIN22<-"Big data make simple (data sets) /////////////////////////////// http://bigdata-madesimple.com/70-websites-to-get-large-data-repositories-for-free/ ///////////////////////////////// " .C6EXPLAIN23<-"AWS Public Datasets /////////////////////////////// Location: https://aws.amazon.com/public-datasets/ AWS hosts a variety of public datasets that anyone can access for free. Previously, large datasets such as the mapping of the Human Genome required hours or days to locate, download, customize, and analyze. Now, anyone can access these datasets via the AWS centralized data repository and analyze them using Amazon EC2 instances or Amazon EMR (Hosted Hadoop) clusters. By hosting this important data where it can be quickly and easily processed with elastic computing resources, AWS hopes to enable more innovation, more quickly. ///////////////////////////////// " .C6EXPLAIN24<-"Bureau of Economic Analysis /////////////////////////////// http://www.bea.gov/ ///////////////////////////////// " .C6EXPLAIN25<-"Wall Street Journal /////////////////////////////// http://www.wsj.com ///////////////////////////////// " .C6EXPLAIN26<-"Options data /////////////////////////////// CBOE stands for Chicago Board of Options Exchange http://cboe.com Equity Options volume http://www.cboe.com/data/avgdailyvolarchive.aspx CBOE Volume & Put/Call Ratios http://www.cboe.com/data/putcallratio.aspx TORQ stands for Trade, Order, Report and Quote database ------------------- The TORQ database contains transactions, quotes, order processing data and audit trail data for a sample of 144 NYSE stocks for the three months November, 1990 through January 1991. This document covers installation, formatting and use of the data. Conceptual and institutional details concerning the data are given in a companion publication Hasbrouck and Sosebee (1992). TORQ database (50M zipped) http://people.stern.nyu.edu/jhasbrou/Research/Working%20Papers/TORQ.zip Manual http://people.stern.nyu.edu/jhasbrou/Research/Working%20Papers/TORQDOC3.PDF ///////////////////////////////// " .C6EXPLAIN27<-"NYSE daily TAQ sample data /////////////////////////////// NYSE daily TAQ sample data NYSE Daily TAQ (Trade and Quote) sample data --------------------------------- Daily TAQ is Extra high frequency data (millisecond-by-millisecond) Location: ftp://ftp.nyxdata.com/Historical%20Data%20Samples/ Index of /Historical Data Samples/Daily TAQ/ Name Size Date Modified [parent directory] EQY_US_ALL_BBO_20031203.zip 366 MB 3/15/16, 12:00:00 AM EQY_US_ALL_BBO_20031204.zip 382 MB 3/14/16, 12:00:00 AM EQY_US_ALL_BBO_20131218.zip 6.4 GB 1/28/14, 12:00:00 AM EQY_US_ALL_BBO_20141030.zip 6.6 GB 11/11/14,12:00:00 AM EQY_US_ALL_BBO_20150805.zip 391 MB 9/16/15, 12:00:00 AM EQY_US_ALL_BBO_20160627_prod.gz 6.3 MB 7/7/16, 12:00:00 AM EQY_US_ALL_BBO_ADMIN_20150805.csv.zip 66.9 MB 8/24/15, 12:00:00 AM EQY_US_ALL_NBBO_20131218.zip 2.0 GB 1/28/14, 12:00:00 AM EQY_US_ALL_NBBO_20150805.zip 3.0 GB 8/24/15, 12:00:00 AM EQY_US_ALL_NBBO_20160627_prod.gz 1.3 MB 7/7/16, 12:00:00 AM EQY_US_ALL_REF_MASTER_20131218.zip 357 kB 1/28/14, 12:00:00 AM EQY_US_ALL_REF_MASTER_20160111.zip 374 kB 3/15/16, 12:00:00 AM EQY_US_ALL_REF_MASTER_20160112.zip 373 kB 3/15/16, 12:00:00 AM EQY_US_ALL_REF_MASTER_PD_20160111.txt 812 kB 3/15/16, 12:00:00 AM EQY_US_ALL_REF_MASTER_PD_20160111.xls 2.7 MB 3/15/16, 12:00:00 AM EQY_US_ALL_REF_MASTER_PD_20160112.txt 812 kB 3/15/16, 12:00:00 AM EQY_US_ALL_REF_MASTER_PD_20160112.xls 2.7 MB 3/15/16, 12:00:00 AM EQY_US_ALL_TRADE_20031203.zip 58.2 MB 3/15/16, 12:00:00 AM EQY_US_ALL_TRADE_20031204.zip 59.6 MB 3/14/16, 12:00:00 AM EQY_US_ALL_TRADE_20131218.zip 298 MB 1/28/14, 12:00:00 AM EQY_US_ALL_TRADE_20141030.zip 271 MB 11/11/14,12:00:00 AM EQY_US_ALL_TRADE_20150805.zip 654 MB 9/16/15, 12:00:00 AM EQY_US_ALL_TRADE_ADMIN_20150805.csv.zip 69.8 MB 8/24/15, 12:00:00 AM http://www.nyxdata.com/Data-Products/Daily-TAQ ///////////////////////////////// " .C6EXPLAIN28<-"NYSE bond data sample /////////////////////////////// Location: ftp://ftp.nyxdata.com/Historical%20Data%20Samples/TAQ%20NYSE%20Bonds/ Index of /Historical Data Samples/TAQ NYSE Bonds/ Name Size Date Modified [parent directory] BND_US_ARCA_BOOK_20130403.csv.gz 36.4 MB 4/29/13, 12:00:00 AM BND_US_ARCA_BOOK_20130404.csv.gz 40.9 MB 4/29/13, 12:00:00 AM BND_US_ARCA_REF_MASTER_20130403.txt 783 kB 4/29/13, 12:00:00 AM BND_US_ARCA_REF_MASTER_20130403.xls 1.9 MB 4/29/13, 12:00:00 AM BND_US_ARCA_REF_MASTER_20130403.xml 3.2 MB 4/29/13, 12:00:00 AM BND_US_ARCA_REF_MASTER_20130403_delta.txt 124 B 4/29/13, 12:00:00 AM BND_US_ARCA_REF_MASTER_20130403_delta.xls 5.0 kB 4/29/13, 12:00:00 AM BND_US_ARCA_REF_MASTER_20130403_delta.xml 633 B 4/29/13, 12:00:00 AM BND_US_ARCA_REF_MASTER_20130404.txt 782 kB 4/29/13, 12:00:00 AM BND_US_ARCA_REF_MASTER_20130404.xls 1.9 MB 4/29/13, 12:00:00 AM BND_US_ARCA_REF_MASTER_20130404.xml 3.2 MB 4/29/13, 12:00:00 AM BND_US_ARCA_REF_MASTER_20130404_delta.txt 1.4 kB 4/29/13, 12:00:00 AM BND_US_ARCA_REF_MASTER_20130404_delta.xls 7.5 kB 4/29/13, 12:00:00 AM BND_US_ARCA_REF_MASTER_20130404_delta.xml 5.5 kB 4/29/13, 12:00:00 AM BND_US_ARCA_TRADE_20130403.csv.gz 1.1 kB 4/29/13, 12:00:00 AM BND_US_ARCA_TRADE_20130404.csv.gz 666 B 4/29/13, 12:00:00 AM BND_US_ARCA_TRADE_BUST_20130403.csv.gz 170 B 4/29/13, 12:00:00 AM BND_US_ARCA_TRADE_BUST_20130404.csv.gz 50 B 4/29/13, 12:00:00 AM ///////////////////////////////// " .C6EXPLAIN29<-"Open Data Inception - 2600+ Open Data Portals Around the World /////////////////////////////// https://opendatainception.io/ Social Network Data Sets -------------------- http://ww35.growmeme.com/overview US Government Web Services and XML Data Sources -------------------------------- http://usgovxml.com/ ///////////////////////////////// " .C6EXPLAIN30<-"Four ways to get data /////////////////////////////// Method #1: download data from Yahoo!fiannce manually Method #2: download data from my website [limited to 20 companies] Method #3: using an R called quandmod [need a fix ] Method #4: using .getYahooDaily() i.e., using quandmod indirectly ///////////////////////////////// " .C6EXPLAIN31<-"fix issues with the R quantmod /////////////////////////////// install.packages('devtools') library(devtools) devtools::install_github('joshuaulrich/quantmod') install.packages('quantmod') library(quantmod) ///////////////////////////////// " .C6EXPLAIN32<-"Bureau of Labor Statistics /////////////////////////////// http://download.bls.gov/ ///////////////////////////////// " .C6EXPLAIN33<-"Quandl database /////////////////////////////// http://quandel.com/ ///////////////////////////////// " .C6EXPLAIN34<-" our .getBSannual() function /////////////////////////////// Step #1:upload two R packages library(tidyverse) library(rvest) Note: if they were not preinstalled, issue the following commands to install them. install.packages(\"tidyverse\") install.packages(\"rvest\") Step #2: Example of using .getBSannual() > x<-.getBSannual(\"ibm\") BS for IBM was successfully downloaded > head(x) Period Ending 12/31/2018 12/31/2017 12/31/2016 12/31/2015 1 Current Assets - - - - 2 Cash And Cash Equivalents 11,379,000 11,972,000 7,826,000 7,686,000 3 Short Term Investments 618,000 608,000 701,000 508,000 4 Net Receivables 8,596,000 10,380,000 10,239,000 9,534,000 5 Inventory 1,682,000 1,583,000 1,553,000 1,551,000 6 Other Current Assets 2,902,000 2,266,000 532,000 293,000 > dim(x) [1] 37 5 ///////////////////////////////// " .C6EXPLAIN35<-"our .getISannual() function /////////////////////////////// Step #1:upload two R packages library(tidyverse) library(rvest) Note: if they were not preinstalled, issue the following commands to install them. install.packages(\"tidyverse\") install.packages(\"rvest\") Step #2: Example of using .getBSannual() > x<-.getBSannual(\"ibm\") BS for IBM was successfully downloaded > head(x) Period Ending 12/31/2018 12/31/2017 12/31/2016 12/31/2015 1 Current Assets - - - - 2 Cash And Cash Equivalents 11,379,000 11,972,000 7,826,000 7,686,000 3 Short Term Investments 618,000 608,000 701,000 508,000 4 Net Receivables 8,596,000 10,380,000 10,239,000 9,534,000 5 Inventory 1,682,000 1,583,000 1,553,000 1,551,000 6 Other Current Assets 2,902,000 2,266,000 532,000 293,000 > dim(x) [1] 37 5 ///////////////////////////////// " .C6EXPLAIN36<-"our .getCFannual() function /////////////////////////////// Step #1:upload two R packages library(tidyverse) library(rvest) Note: if they were not preinstalled, issue the following commands to install them. install.packages(\"tidyverse\") install.packages(\"rvest\") Step #2: Example of using .getBSannual() > x<-.getBSannual(\"ibm\") BS for IBM was successfully downloaded > head(x) Period Ending 12/31/2018 12/31/2017 12/31/2016 12/31/2015 1 Current Assets - - - - 2 Cash And Cash Equivalents 11,379,000 11,972,000 7,826,000 7,686,000 3 Short Term Investments 618,000 608,000 701,000 508,000 4 Net Receivables 8,596,000 10,380,000 10,239,000 9,534,000 5 Inventory 1,682,000 1,583,000 1,553,000 1,551,000 6 Other Current Assets 2,902,000 2,266,000 532,000 293,000 > dim(x) [1] 37 5 ///////////////////////////////// " .C6EXPLAIN37<-" our R data sets /////////////////////////////// http://datayyy.com/data_R http://datayyy.com/data_R/list.txt ///////////////////////////////// " .C6EXPLAIN38<-" our CSV data sets /////////////////////////////// http://datayyy.com/data_csv http://datayyy.com/data_csv/list.txt ///////////////////////////////// " .C6EXPLAIN39<-" our Excel files /////////////////////////////// http://datayyy.com/data_excel http://datayyy.com/data_excel/list.txt ///////////////////////////////// " .C6EXPLAIN40<-"our text files /////////////////////////////// http://datayyy.com/data_txt http://datayyy.com/data_txt/list.txt ///////////////////////////////// "