probability values from vector in R

https://datascience.stackexchange.com/questions/37329/how-to-convert-an-array-of-numbers-into-probability-values

left_join with NA in R dplyr

If you left_join by a variable with NAs you will get those NA rows. This is annoying. So use

na_matches = "never"

Table of useful R packages for data science

https://www.computerworld.com/article/2921176/business-intelligence/great-r-packages-for-data-import-wrangling-visualization.html

 

My favorite R packages for data visualization and munging

PACKAGE	CATEGORY	DESCRIPTION	SAMPLE USE	AUTHOR
dplyr	data wrangling, data analysis	The essential data-munging R package when working with data frames. Especially useful for operating on data by categories. CRAN.	See the intro vignette	Hadley Wickham
purrr	data wrangling	purrr makes it easy to apply a function to each item in a list and return results in the format of your choice. It’s more complex to learn than the older plyr package, but also more robust. And, its functions are more standardized than base R’s apply family — plus it’s got functions for tasks like error-checking. CRAN.	map_df(mylist, myfunction) More: Charlotte Wickham’s purr tutorial video, the purrr cheat sheet PDF download.	Hadley Wickham
readxl	data import	Fast way to read Excel files in R, without dependencies such as Java. CRAN.	read_excel(“my-spreadsheet.xls”, sheet = 1)	Hadley Wickham
googlesheets	data import, data export	Easily read data into R from Google Sheets. CRAN.	mysheet <- gs_title(“Google Spreadsheet Title”) mydata <- mydata <- gs_read(mysheet, ws = “WorksheetTitle”)	Jennifer Bryan
readr	data import	Base R handles most of these functions; but if you have huge files, this is a speedy and standardized way to read tabular files such as CSVs into R data frames, as well as plain text files into character strings with read_file. CRAN.	read_csv(myfile.csv)	Hadley Wickham
rio	data import, data export	rio has a good idea: Pull a lot of separate data-reading packages into one, so you just need to remember 2 functions: import and export. CRAN.	import(“myfile”)	Thomas J. Leeper & others
Hmisc	data analysis	There are a number of useful functions in here. Two of my favorites: describe, a more robust summary function, and Cs, which creates a vector of quoted character strings from unquoted comma-separated text. Cs(so, it, goes) creates c(“so”, “it”, “goes”). CRAN.	describe(mydf) Cs(so, it, goes)	Frank E Harrell Jr & others
datapasta	data import	Data copy and paste: Meet reproducible research. If you’ve copied data from the Web, a spreadsheet, or other source into your clipboard, datapasta lets you paste it into R as an R object, with the code to reproduce it. It includes RStudio add-ins as well as command-line functions for transposing data, turning it into markdown format, and more. CRAN.	df_paste() to create a data frame, vector_paste() to create a vector.	Miles McBain
sqldf	data wrangling, data analysis	Do you know a great SQL query you’d use if your R data frame were in a SQL database? Run SQL queries on your data frame with sqldf. CRAN.	sqldf(“select * from mydf where mycol > 4”)	G. Grothendieck
jsonlite	data import, data wrangling	Parse json within R or turn R data frames into json. CRAN.	myjson <- toJSON(mydf, pretty=TRUE) mydf2 <- fromJSON(myjson)	Jeroen Ooms & others
XML	data import, data wrangling	Many functions for elegantly dealing with XML and HTML, such as readHTMLTable. CRAN.	mytables <- readHTMLTable(myurl)	Duncan Temple Lang
httr	data import, data wrangling	An R interface to http protocols; useful for pulling data from APIs. See the httr quickstart guide. CRAN.	r <- GET(“http://httpbin.org/get&#8221;) content(r, “text”)	Hadley Wickham
quantmod	data import, data visualization, data analysis	Even if you’re not interested in analyzing and graphing financial investment data, quantmod has easy-to-use functions for importing economic as well as financial data from sources like the Federal Reserve. CRAN.	getSymbols(“AITINO”, src=”FRED”)	Jeffrey A. Ryan
tidyquant	data import, data visualization, data analysis	Another financial package that’s useful for importing, analyzing and visualizing data, integrating aspects of other popular finance packages as well as tidyverse tools. With thorough documentation. CRAN.	aapl_key_ratios <- tq_get(“AAPL”, get = “key.ratios”)	Matt Dancho
rvest	data import, web scraping	Web scraping: Extract data from HTML pages. Inspired by Python’s Beautiful Soup. Works well with Selectorgadget. CRAN.	See the package vignette	Hadley Wickham
tidyr	data wrangling	While I still prefer the older reshape2package for some general re-arranging, tidyr won me over with specialized functions like fill (fill in missing columns from data above) and replace_na. Its main purpose is helping you change data row and column formats from “wide” to “long”. CRAN.	See examples in this blog post.	Hadley Wickham
splitstackshape	data wrangling	It’s rare that I’d recommend a package that hasn’t been updated in years, but the cSplit() function solves a rather complex shaping problem in an astonishingly easy way. If you have a data frame column with one or more comma-separated values (think a survey question with “select all that apply”), this is worth an install if you want to separate each item into its own new data frame row.. CRAN.	cSplit(mydata, “multi_val_column”, sep = “,”, direction = “long”).	Ananda Mahto
magrittr	data wrangling	This package gave us the %>% symbol for chaining R operations, but it’s got other useful operators such as %<>% for mutating a data frame in place and and . as a placeholder for the original object being operated upon. CRAN.	mydf %<>% mutate(newcol = myfun(colname))	Stefan Milton Bache & Hadley Wickham
validate	data wrangling	Intuitive data validation based on rules you can define, save and re-use. CRAN.	See the introductory vignette.	Mark van der Loo & Edwin de Jonge
testthat	programming	Package that makes it easy to write unit tests for your R code. CRAN.	See the testing chapter of Hadley Wickham’s book on R packages.	Hadley Wickham
data.table	data wrangling, data analysis	Popular package for heavy-duty data wrangling. While I typically prefer dplyr, data.table has many fans for its speed with large data sets. CRAN.	Useful tutorial	Matt Dowle & others
stringr	data wrangling	Numerous functions for text manipulation. Some are similar to existing base R functions but in a more standard format, including working with regular expressions. Some of my favorites: str_pad and str_trim. CRAN.	str_pad(myzipcodevector, 5, “left”, “0”)	Hadley Wickham
lubridate	data wrangling	Everything you ever wanted to do with date arithmetic, although understanding & using available functionality can be somewhat complex. CRAN.	mdy(“05/06/2015”) + months(1) More examples in the package vignette	Garrett Grolemund, Hadley Wickham & others
zoo	data wrangling, data analysis	Robust package with a slew of functions for dealing with time series data; I like the handy rollmean function with its align=right and fill=NA options for calculating moving averages. CRAN.	rollmean(mydf, 7)	Achim Zeileis & others
editR	data display	Interactive editor for R Markdowndocuments. Note that R Markdown Notebooks are another useful way to generate Markdown interactively. editR is on GitHub.	editR(“path/to/myfile.Rmd”)	Simon Garnier
knitr	data display	Add R to a markdown document and easily generate reports in HTML, Word and other formats. A must-have if you’re interested in reproducible research and automating the journey from data analysis to report creation. CRAN.	See the Minimal Examples page.	Yihui Xie & others
officeR	data display	Import and edit Microsoft Word and PowerPoint documents, making it easy to add R-generated analysis and visualizations to existing as well as new reports and presentations. CRAN.	my_doc <- read_docx() %>% body_add_img(src = myplot) The package website has many more examples.	David Gohel
listviewer	data display, data wrangling	While RStudio has since added a list-viewing option, this HTML widget still offers an elegant way to view complex nested lists within R. GitHub timelyportfolio/listviewer.	jsonedit(mylist)	Kent Russell
DT	data display	Create a sortable, searchable table in one line of code with this R interface to the jQuery DataTables plug-in. GitHub rstudio/DT.	datatable(mydf)	RStudio
ggplot2	data visualization	Powerful, flexible and well-thought-out dataviz package following ‘grammar of graphics’ syntax to create static graphics, but be prepared for a steep learning curve. CRAN.	qplot(factor(myfactor), data=mydf, geom=”bar”, fill=factor(myfactor)) See my searchable ggplot2 cheat sheet and time-saving code snippets.	Hadley Wickham
patchwork	data visualization	Easily combine ggplot2 plots and keep the new, merged plot a ggplot2 object. plot_layout() adds ability to set columns, rows, and relative sizes of each component graphic. GitHub.	plot1 + plot2 + plot_layout(ncol=1)	Thomas Lin Pedersen
ggiraph	data visualization	Make ggplot2 plots interactive with this extension’s new geom functions such geom_bar_interactive and arguments for tooltips and JavaScript onclicks. CRAN.	g <- ggplot(mpg, aes( x = displ, y = cty, color = drv) ) my_gg <- g + geom_point_interactive(aes(tooltip = model), size = 2) ggiraph(code = print(my_gg), width = .7).	David Gohel
dygraphs	data visualization	Create HTML/JavaScript graphs of time series – one-line command if your data is an xts object. CRAN.	dygraph(myxtsobject)	JJ Allaire & RStudio
googleVis	data visualization	Tap into the Google Charts API using R. CRAN.	mychart <- gvisColumnChart(mydata) plot(Column) Numerous examples here	Markus Gesmann & others
metricsgraphics	data visualization	R interface to the metricsgraphics JavaScript library for bare-bones line, scatterplot and bar charts. GitHub hrbrmstr/metricsgraphics.	See package intro	Bob Rudis
taucharts	data visualization	This html widget library is especially useful for scatterplots where you want to view multiple regression options. However, it does much more than that, including line and bar charts with legends and tooltips. GitHub hrbrmstr/taucharts.	See the author’s post on RPubs	Bob Rudis
RColorBrewer	data visualization	Not a designer? RColorBrewer helps you select color palettes for your visualizations. CRAN. Note: For even more palettes, check out packages viridis for colors that print well in greyscale and are easier to read if you’re color blind, pals, rcartcolor for map colors, colorr for sports-team colors, nord for “Northern-themed Color palettes,” and wesanderson for color schemes used by director Web Anderson.	See Jennifer Bryan’s tutorial	Erich Neuwirth
sf	mapping, data wrangling	This package makes it much easier to do GIS work in R. Simple features protocols make geospatial data look a lot like regular data frames, while various functions allow for analysis such as determining whether points are in a polygons. A GIS game-changer for R. CRAN.	See the package vignettes, starting with the introduction, Simple Features for R.	Edzer Pebesma & others
leaflet	mapping	Map data using the Leaflet JavaScript library within R. GitHub rstudio/leaflet.	See my tutorial	RStudio
ggmap	mapping	Although I don’t use this package often for its main purpose of pulling down background map tiles, it’s my go-to for geocoding up to 2,500 addresses with the Google Maps API with its geocode and mutate_geocode functions. CRAN.	geocode(“492 Old Connecticut Path, Framingham, MA”)	David Kahle &Hadley Wickham
tmap & tmaptools	mapping	These package offer an easy way to read in shape files and join data files with geographic info, as well as do some exploratory mapping. Recent functionality adds support for simple features, interactive maps and creating leaflet objects. Plus, tmaptools::palette_explorer() is a great tool for picking ColorBrewer palettes. CRAN.	See the package vignette or my mapping in R tutorial	Martijn Tennekes
mapsapi	mapping, data wrangling	This interface to the Google Maps Direction and Distance Matrix APIs let you analyze and map distances and driving routes. CRAN.	google_directions( origin = c(my_longitude, my_latitude), destination = c(my_address), alternatives = TRUE Also see the vignette	Michael Dorman
tidycensus	mapping, data wrangling	Want to analyze and map U.S. Census Bureau data from 5-year American Community Surveys or 10-year censuses? This makes it easy to download numerical and geospatial info in R-ready format. CRAN.	See Basic usage of tidycensus.	Kyle E. Walker
glue	data wrangling	Main function, also glue, evaluates variables and R expressions within a quoted string, as long as they’re enclosed by {} braces. This makes for an elegant paste() replacement. CRAN.	glue(“Today is {Sys.Date()}”)	Jim Hester
rga	Web analytics	Use Google Analytics with R. GitHub skardhamar/rga.	See package README file and my tutorial	Bror Skardhamar
RSiteCatalyst	Web analytics	Use Adobe Analytics with R. GitHub randyzwitch/RSiteCatalyst.	See intro video	Randy Zwitch
roxygen2	package development	Useful tools for documenting functions within R packages. CRAN.	See this short, easy-to-read blog post on writing R packages	Hadley Wickham & others
shiny	data visualization	Turn R data into interactive Web applications. I’ve seen some nice (if sometimes sluggish) apps and it’s got many enthusiasts. CRAN.	See the tutorial	RStudio
flexdashboard	data visualization	If Shiny is too complex and involved for your needs, this package offers a simpler (if somewhat less robust) solution based on R Markdown. CRAN.	More info in Using flexdashboard	JJ Allaire, RStudio & others
openxlsx	misc	If you need to write to an Excel file as well as read, this package is easy to use. CRAN.	write.xlsx(mydf, “myfile.xlsx”)	Alexander Walker
gmodels	data wrangling, data analysis	There are several functions for modeling data here, but the one I use, CrossTable, simply creates cross-tabs with loads of options — totals, proprotions and several statistical tests. CRAN.	CrossTable(myxvector, myyvector, prop.t=FALSE, prop.chisq = FALSE)	Gregory R. Warnes
janitor	data wrangling, data analysis	Basic data cleaning made easy, such as finding duplicates by multiple columns, making R-friendly column names and removing empty columns. It also has some nice tabulating tools, like adding a total row, as well as generating tables with percentages and easy crosstabs. CRAN.	tabyl(mydf, sort = TRUE) %>% adorn_totals(“row”)	Samuel Firke
car	data wrangling	car’s recode function makes it easy to bin continuous numerical data into categories or factors. While base R’s cut accomplishes the same task, I find recode’s syntax to be more intuitive – just remember to put the entire recoding formula within double quotation marks. dplyr’s case_when() function is another option worth considering. CRAN.	recode(x, “1:3=’Low’; 4:7=’Mid’; 8:hi=’High'”)	John Fox & others
rcdimple	data visualization	R interface to the dimple JavaScript library with numerous customization options. Good choice for JavaScript bar charts, among others. GitHub timelyportfolio/rcdimple.	dimple(mtcars, mpg ~ cyl, type = “bar”)	Kent Russell
scales	data wrangling	While this package has many more sophisticated ways to help you format data for graphing, it’s worth a download just for the comma(), percent() and dollar() functions. CRAN.	comma(mynumvec)	Hadley Wickham
plotly	data visualization	R interface to the Plotly JavaScript library that was open-sourced in late 2015. Basic graphs have a distinctive look which may not be for everyone, but it’s full-featured, relatively easy to learn (especially if you know ggplot2) and includes a ggplotly() function to turn graphs created with ggplot2 interactive. CRAN.	d <- diamonds[sample(nrow(diamonds), 1000), ] plot_ly(d, x = carat, y = price, text = paste(“Clarity: “, clarity), mode = “markers”, color = carat, size = carat)	Carson Sievert & others
highcharter	data visualization	R wrapper for the robust and well documented Highcharts JavaScript library, one of my favorite choices for presentation-quality interactive graphics. The package uses ggplot2-like syntax, including options for handling both long and wide data, and comes with plenty of examples. Note that a paid Highcharts license is needed to use this for commercial or government work (it’s free for personal and non-profit projects). CRAN. . CRAN.	hchart(mydf, “charttype”, hcaes(x = xcol, y = ycol, group = groupbycol))	Joshua Kunst & others
profvis	programming	Is your R code sluggish? This package gives you a visual representative of your code line by line so you can find the speed bottlenecks. CRAN.	profvis({ your code here })	Winston Chang & others
tidytext	text mining	Elegant implementation of text mining functions using Hadley Wickham’s “tidy data” principles. CRAN.	See tidytextmining.com for numerous examples.	Julia Silge & David Robinson
diffobj	data analysis	Base R’s identical() function tells you whether or not two objects are the same; but if they’re not, it won’t tell you why. diffobj gives you a visual representation of how two R objects differ. CRAN.	diffObj(x,y)	Brodie Gaslam & Michael B. Allen
Prophet	forecasting	I don’t do much forecasting analysis; but if I did, I’d start with this package. CRAN.	See the Quick start guide.	Sean Taylor & Ben Letham at Facebook
feather	data import, data export	This binary data-file format can be read by both Python and R, making data interchange easier between the two languages. It’s also built for I/O speed. CRAN.	write_feather(mydf, “myfile”)	Wes McKinney & Hadley Wickham
fst	data import, data export	Another alternative for binary file storage (R-only), fst was built for fast storage and retrieval, with access speeds above 1 GB/sec. It also offers compression that doesn’t slow data access too much, as well as the ability to import a specific range of rows (by row number). CRAN.	write.fst(mydf, “myfile.fst”, 100)	Mark Klik
googleAuthR	data import	If you want to use data from a Google API in an R project and there’s not yet a specific package for that API, this is the place to turn for authenticating CRAN.	See examples on the package website and this gist for use with Google Calendars. CRAN.	Mark Edmondson
devtools	package development, package installation	devtools has a slew of functions aimed at helping you create your own R packages, such as automatically running all example code in your help files to make sure everything works. Requires Rtools on Windows and XCode on a Mac. On CRAN.	run_examples()	Hadley Wickham & others
remotes	package installation	If you want to install R packages from GitHub, devtools was long the go-to. However, it has a ton of other functions and some hefty dependences. remotes is a lighter-weight alternative if all you want is to install packages from GitHub as well as Bitbucket and some other sources. CRAN. (ghit is another option, but is GitHub-only.)	remotes::install_github(“mangothecat/franc”)	Gabor Csardi & others
githubinstall	package installation	Do you want to install a package from GitHub without typing out the GitHub user name along with the repo name? Whether because you can’t remember a package’s GitHub owner’s name, that name is long/complex to type out, or you just want to save yourself a little typing, this package is a handy option. Simply run githubinstall(“packagename”) and the package will suggest an account; then you respond Y to install or n if it’s the wrong one. It even includes fuzzy matching if you misspell a package name!	githubinstall::githubinstall::(“AnomalyDetection”)	Koji Makiyama
installr	misc	Windows only: Update your installed version of R from within R. On CRAN.	updateR()	Tal Galili & others
reinstallr	misc	Seeks to find packages that had previously been installed on your system and need to be re-installed after upgrading R. CRAN.	reinstallr()	Calli Gross
usethis	package development, programming	Initially aimed at package development, usethis now includes useful functions for any coding project. Among its handy features are an edit family that lets you easily update your .Renvironment and .Rprofile files. On CRAN, but install GitHub version from “r-lib/usethis” for latest updates.	edit_r_environ()	Hadley Wickham, Jennifer Bryan & RStudio
here	misc	This package has one function with a single, useful purpose: find your project’s working directory. Surprisingly helpful if you want your code to run on more than one system. CRAN.	my_project_directory <- here()	Kirill Müller
pacman	misc, package installation	This package is another that aims to solve one problem, and solve it well: package installation. The main functions will loadi a package that’s already installed or installing it first if it’s not available. While this is certainly possible to do with base R’s require() and an if statement, p_load() is so much more elegant for CRAN packages, or p_load_gh() for GitHub. Other useful options include p_temp(), which allows for a temporary, this-session-only package installation. CRAN.	p_load(dplyr, here, tidycensus)	Tyler Rinker
plumber	data export, programming	Turn any R function into a host-able API with a line or two of code. This well-thought-out package makes it easy to use R for data handling in other, non-R coding projects. CRAN.	See the documentation or my article Create your own Slack bots — and Web APIs — with R	Jeff Allen, Trestle Technology & others
echarts4r	data visualization	R wrapper for the powerful and flexible ECharts JavaScript library. It features dozens of chart and graph types, from bar and line charts to sunbursts, heat maps, and geographical maps. Hundreds of customizations not explicitly mentioned in the package docs are nevertheless available; you just need to peruse the original ECharts documentation. (ECharts is an Apache Software Foundation incubator project.)	mtcars %>% e_charts(wt) %>% e_line(mpg)	John Coene
cloudyR project	data import, data export	This is a collection of packages aimed at making it easier for R to work with cloud platforms such as Amazon Web Services, Google and Travis-CI. Some are already on CRAN, some can be found on GitHub.	See the list of packages.	Various
geofacet	data visualization, mapping	To be honest, I rarely need the ability create “geofacets” — maps with same-sized blocks in geospatially appropriate locations. However, this package is so cool that I had to include it. Geofaceting is best understood by looking at an example. The package lets you create your own geofacet visualizations using ggplot2 and built-in grids such as US states, EU countries and San Francisco Bay Area counties. Even more impressive, it comes with design-your-own geofacet grid capabilities. CRAN.	grid_design()	Ryan Hafen
reticulate	programming	If you know Python as well as R, this package offers a suite of tools for calling Python from within R, as well as “translating” between R and Python objects such as Pandas data frames and R data frames. CRAN.	See the reticulate package website.	JJ Allaire
beepr	misc	This is pretty much pure fun. Yes, getting an audible notification when code finishes running or encounters an error could be useful; but here, the available sounds include options like a fanfare flourish, a Mario Brothers tune, and even a scream. CRAN.	beep(“wilhelm”)	Rasmus Bååth

A few important points for newbies:

make yaml into a data frame in R

y1l <- yaml::yaml.load_file("y1.yaml")

plyr::ldply (y1l, data.frame, stringsAsFactors=FALSE)

https://rstudio-pubs-static.s3.amazonaws.com/122299_a69a7028271b46bf99d167485f0e821e.html

github desktop will not open from task bar

refocus the window using alt+space

https://github.com/desktop/desktop/issues/3757#issuecomment-366373497

the difference in hours between times in google sheets

You can watch the video, but just make your times numeric, subtract and multiply by 24. You will need to use the format tab in sheets, below is the formula formatNumeric() is that step.

(formatNumeric(clock out) - formatNumeric(clock in)) * 24

 

 

Filestream Memory Problem Full

remove all temp files windows+R

then type

explorer

%temp%

temp

prefetch

 

delete them files!

 

BitLocker Could Not Be Enabled: 5 Ways to Fix This Error

Store Copy file or folder in multiple places or locations google drive

click on object and shift+z

r remove all objects except

rm(list=setdiff(ls(), "x"))

https://stackoverflow.com/questions/6190051/how-can-i-remove-all-objects-but-one-from-the-workspace-in-r

extract numeric number from character string

library(stringr)
str_extract(data, "[[:digit:]]+")

 

https://stackoverflow.com/questions/15451251/extract-numeric-part-of-strings-of-mixed-numbers-and-characters-in-r