For the past eight years, climate science has been under a sort of spell in the House of Representatives. Instead of trying to understand it better or even acknowledging some of the field’s current uncertainties, House Science Committee Chairman Rep. Lamar Smith (R-Texas) used his position to harass federal climate scientists with subpoenas while holding hearings on “Making the EPA Great Again” or whether “global warming theories are alarmist” and researchers are pursuing a “personal agenda.”
But Smith retired this year and Democrats won control of the House on Tuesday. Now some on Capitol Hill say that the anti-climate science spell may be broken.
Both Republicans and Democrats woke up Wednesday morning claiming victory in Tuesday’s midterms. Democrats patted themselves on the back for taking back the House of Representatives and for flipping seven governorships from red to blue. And in a press conference, President Donald Trump praised his party, and himself, for gaining ground in the Senate.
Americans remain sharply divided at the ballot box, from which political party they support to initiatives on issues like climate change. But they consistently voted against one thing on Tuesday: gerrymandering. In Michigan, Missouri, and Colorado, voters overwhelmingly passed ballot initiatives to put an end to this practice; as of Wednesday evening, another in Utah held onto a slight lead. They join another initiative passed in Ohio earlier this year.
There is perhaps no greater example of Silicon Valley’s soft power than watching a debate around a grassroots proposal to fight homelessness transform into a Twitter war between tech billionaires and their preferred form of taxation. Rob Reich, author of the new book Just Giving: Why Philanthropy Is Failing Democracy and How It Can Do Better, called the drama around San Francisco’s Proposition C just as much a sign of our absurd times as New York governor Andrew Cuomo’s offer to rename himself “Amazon Cuomo” if it would inspire Jeff Bezos to put Amazon’s second headquarters in New York.
Even in the midst of a tech backlash, “the deference to economic power is still strong” among elected officials, says Reich, a Stanford political science professor. But on Tuesday, voters most familiar with Big Tech’s longterm impact refused to capitulate. Nearly 60 percent of San Francisco voters supported Prop C, which is projected to double the city’s budget for homeless services, raising an additional $300 million a year through a gross receipts tax on roughly 400 companies, including Square and Salesforce, whose billionaire CEOs bickered online about the initiative.
Investigators are still working to discover exactly what went wrong with a Lion Air flight in Indonesia on Monday, October 29, when a Boeing 737 MAX plunged into the Java Sea, killing all 189 people onboard.
But the initial findings have highlighted a possible sensor problem, and that has been enough for Boeing to issue safety warnings to all the airlines that operate those planes, telling pilots to brush up on how to deal with confusing readings or erratic actions from the flight control computer, which could cause the planes to dive, hard. And now the FAA says it’s throwing its weight behind Boeing’s advisory, to make it mandatory for US airlines to comply.
On Monday morning, just 24 hours before polls opened in the US midterm elections, President Trump sounded an alarm with a Tweet: “Law Enforcement has been strongly notified to watch closely for any ILLEGAL VOTING which may take place in Tuesday’s Election (or Early Voting). Anyone caught will be subject to the Maximum Criminal Penalty allowed by law. Thank you!”
The rumor was part of a pair; over the weekend, Trump tweeted that Indiana senator Joe Donnelly was “trying to steal the election” by buying Facebook ads for the libertarian Senate candidate.
Tuesday’s midterms don’t shed much light on the future of net neutrality. But advocates do see rays of hope shining through the fog of uncertainty.
Democrats, who generally favor rules barring internet service providers like Comcast and Verizon from blocking or otherwise discriminating against content, took control of the House. And even after losing ground in the Senate, the party is tantalizingly close to having enough support from Senate Republicans to pass new net neutrality protections.
We’ve developed an energy-based model that can quickly learn to identify and generate instances of concepts, such as near, above, between, closest, and furthest, expressed as sets of 2d points. Our model learns these concepts after only five demonstrations. We also show cross-domain transfer: we use concepts learned in a 2d particle environment to solve tasks on a 3-dimensional physics-based robot.
Many hallmarks of human intelligence, such as generalizing from limited experience, abstract reasoning and planning, analogical reasoning, creative problem solving, and capacity for language require the ability to consolidate experience into concepts, which act as basic building blocks of understanding and reasoning. Our technique enables agents to learn and extract concepts from tasks, then use these concepts to solve other tasks in various domains. For example, our model can use concepts learned in a two-dimensional particle environment to let it carry out the same task on a three-dimensional physics-based robotic environment – without retraining in the new environment.
A simulated robot trained via an energy-based model navigates its arm to be between two points, using a concept learned in a different 2D domain.
This work uses energy functions to let our agents learn to classify and generate simple concepts, which they can use to solve tasks like navigating between two points in dissimilar environments. Examples of concepts include visual (“red” or “square”), spatial (“inside”, “on top of”), temporal (“slow”, “after”), social (“aggressive”, “helpful”) among others. These concepts, once learned, act as basic building blocks of agent’s understanding and reasoning, as shown in other research from DeepMind and Vicarious.
Energy functions let us build systems that can generate (left) and also identify (right) basic concepts, like the notion of a square.
Energy functions work by encoding a preference over states of the world, which allows an agent with different available actions (changing torque vs directly changing position) to learn a policy that works in different contexts – this roughly translates to the development of a conceptual understanding of simple things.
To create the energy function, we mathematically represent concepts as energy models. The idea of energy models is rooted in physics, with the intuition that observed events and states represent low-energy configurations.
We define an energy function E(x, a, w) for each concept in terms of:
A continuous-valued vector (w), used as conditioning, that specifies the concept for which energy is being calculated
States of the world are composed of sets of entities and their properties and positions (like the dots below, which have both positional and colored properties). Attention masks, used for “identification”, represent a model’s focus on some set of entities. The energy model outputs a single positive number indicating whether the concept is satisfied (when energy is zero) or not (when energy is high). A concept is satisfied when an attention mask is focused on a set of entities that represent a concept, which requires both that the entities are in the correct positions (modification of x, or generation) and that the right entities are being focused on (modification of a, or identification).
We construct the energy function as a neural network based on the relational network architecture, which allows it to take an arbitrary number of entities as input. The parameters of this energy function are what is being optimized by our training procedure; other functions are derived implicitly from the energy function.
This approach lets us use energy functions to learn a single network that can perform both generation and recognition. This allows us to cross-employ concepts learned from generation to identification, and vice versa. (Note: This effect is already observed in animals via mirror neurons.)
Our training data is composed of trajectories of (attention mask, state), which we generate ahead of time for the specific concepts we’d like our model to learn. We train our model by giving it a set of demonstrations (typically 5) for a given concept set, and then give it a new environment (X0) and ask it to predict the next state (X1) and next attention mask (a). We optimize the energy function such that the next state and next attention mask found in the training data are assigned low energy values. Similar to generative models like variational autoencoders, the model is incentivized to learn values that usefully compress aspects of the task. We trained our model using a variety of concepts involving, visual, spatial, proximal, and temporal relations, and quantification in a two-dimensional particle environment.
Spatial Region Concepts: given demonstration 2D points (left), energy function over point placement is inferred (middle), stochastic gradient descent over energy is then used to generate new points (right)
We evaluated our approach across a suite of tasks designed to see how well our single system could learn to identify and generate things united by the same concept; our system can learn to classify and generate specific sets of spatial relationships, or can navigate entities through a scene in a specific way, or can develop good judgements for concepts like quantity (one, two, three, or more than three) or proximity.
Quantity Concept: demonstration attention is placed on one, two, three, or more than three entities. Inference is used to generate attention masks of similar quantity
Models perform better when they can share experience between learning to generate concepts (by moving entities within the state vector x) and identify them (by changing the attention mask over a fixed state vector): when we evaluated models trained on both of these operations, they performed better on each single operation than models trained only on that single operation alone. We also discovered indications of transfer learning – an energy function trained only on a recognition context performs well on generation, even without being explicitly trained to do so.
Proximity Concepts: demonstration events bring attention to the entity closest or furthest to the marker or to bring the marker to be closest or furthest to entity of a particular color (left). Inference is used to generate attention masks for closest or further entity (recognition) or to place the marker to be closest or furthest from an entity (generation) (right)
In the future we’re excited to explore a wider variety of concepts learned in richer, three-dimensional environments, integrate concepts with the decision-making policies of our agents (we have so far only looked at concepts as things learned from passive experience), and explore connections between concepts and language understanding. If you are interested in this line of research, consider working at OpenAI!
Thanks to those who contributed to this paper and blog post:
Blog post: Prafulla Dhariwal, Alex Nichol, Alec Radford, Yura Burda, Jack Clark, Greg Brockman, Ilya Sutskever, Ashley Pilipiszyn
As a monitoring company, time-series databases (TSDBs) are the foundation we rely on to build a scalable and reliable monitoring solution
at Outlyer. Over the past few years we’ve gone through 3 iterations of our architecture as we scaled up with more metrics
and more customers, and we haven’t always gotten it right. We’ve tested most of the open-source TSDBs
out there, and currently own the top ranked blog on the subject, and also attempted to build our own.
You could say, through a lot of trial and error we’ve become experts in building TSDBs, and know how hard it is to do at scale. This
is why I wanted to share our journey, and as every week it seems another person has released their own open-source TSDB, maybe help
some of you avoid the mistakes we made along the way.
This blog is based on a talk I did at our last DOXSFO meetup so you can either watch the video
or continue reading below:
Our Journey To Building Something That Works & Scales
Version 1: 2013-2015
When we started back in 2013, analytics and containers seemed quite far off on the horizon. With a very small team we opted to go for the simplest solution
of just storing metrics and datapoints. We knew it had to be reliable and performant, and as we had a small team, having something operationaly resilient and
easy to manage was important to us.
That lead us to building our first architecture on Riak. Although Riak is a
simple key/value store, Riak Core, built in Erlang, is very reliable and performant. During testing we literally killed nodes and replaced them,
and Riak “magically” redistributed the data and continued running without any outages.
This worked well for the first year as we built the first version of Outlyer, but as we added more users we started getting performance issues, so we did
what all good engineering teams do when hitting scaling issues, and put the in-memory cache Redis
to act as a write buffer and quicker source to query for the latest metrics that were coming in.
Version 2: 2015-2017
Version 1 actually worked really well for the first year and a half, but by 2015 the limitations of this approach were becoming obvious as more
advanced analytics and containers became a fast approaching reality, and our product would need to advance towards having more complex analytics
capabilities and dimensional metrics.
Riak out of the box, being only a key/value store, only allowed us to do simple, non-dimensional metrics (metric names with no ability to label metrics),
and we knew we’d need to add this capability, as well as a query layer that would enable us to do advanced queries such as adding and dividing time-series
together, calculating percentiles on the fly, and other common queries you’d expect from a TSDB.
We had a choice of either building it ourselves on top of Riak or bringing in another solution already out there. We looked at the obvious
solutions at the time like InfluxDB, as we knew the founder Paul and liked it a lot,
but at the time (2015) its lack of clustering and maturity didn’t convince us we could use it for our needs (they’ve made huge progress since).
Along the way, we stumbled across a TSDB created by the talented Heinz Gies,
called DalmatinerDB. Heinz is an extremely smart and accomplished Erlang programmer, and had
taken the best part we liked about Riak, Riak core, and built a TSDB around it with a query engine that could do advanced analytics on top. It didn’t
support dimensional metrics when we found it, but as the team were excited to use Erlang based on WhatsApp’s success at scale).
Our initial testing showed a single node could write 3 million metrics/second. We felt confident
we could take DalmatinerDB, make it our own and become the main contributors to the project. We even contracted Heinz in to help, essentially sponsoring its
development, and worked closely with him to add a new PostgreSQL index so we could support dimensional labels on metrics.
This is where our journey stumbled. We continued to grow and add some new large customers, and as DalmatinerDB had never operated at the scale we were at, all the
non-obvious issues of building a TSDB started to appear quickly, and we hit some pretty bad performance issues that we ultimately fixed, but only by putting our
entire engineering resources onto the problem. Like an onion, as soon as we fixed one performance problem, the next layer would reveal another issue underneath.
Without going into the full detail of the challenges we hit, here is a short summary:
No Sensible Limits: There were no real limits or protections around what customers could send us in our architecture. This really hit us when one team at
one of our customers decided to dump 30 million metrics on us, send all of their mobile product metrics into Outlyer. This immediately put our team into
fire-fighting mode trying to operate at another level of scale we weren’t expecting to hit until much later.
Container Churn: As containers became more prevailent in 2016, we started to see some container environments churning containers every 6 minutes in some cases
leading to the problem of Series Churn (see more below) and our PostgreSQL index exploded as it had to index more metrics, slowing down our ability to query for metrics.
Conflicting Write & Read Architectures: While we were blown away with being able to write 3 million metrics/second on DalmatinerDB in our initial testing, the challenge
any TSDB needs to handle, is balancing the ideal patterns for writing and reading data in parallel as scale. Fabian Reinartz
from Prometheus put it elegantly:
“There’s obviously a strong tension between the ideal pattern for writing collected data to disk and the layout that would be significantly more efficient
for serving queries. It is the fundamental problem our TSDB has to solve.”
We experienced this problem a lot on DalmatinerDB as we scaled. While we were able to write metrics at scale, the same architecture that enabled us to achieve such
high write volumes, really hurt us when we tried to query the data back to users. For example, DalmatinerDB didn’t push queries down to the nodes, which meant
it had to pull all the datapoints for a query across our network to the query engine, which would then cacluate the time-series the user saw based on the query. This meant that
we started seeing network IO bottlenecks as at times 40GB of raw datapoints would be pulled in for a single query!
In the end, after investing a lot of time and resources, we did the other engineering solution that teams take to solve scaling issues, and essentially sharded all our largest
customers onto seperate DalmtainerDB clusters. This stopped the fires, but we knew that unless we were going to become a time-series database company, we would have to
fundamentally change our approach, instead of trying to peel the onion and hitting all the other unknown scaling issues we would inevitably hit as we grew larger.
The good news is during this time, as we were also writing and helping design the TSDB, we had learned the hard way how to scale and operate a TSDB, and could take
that knowledge to do something that would actually work for us.
Why Building a TSDB is Hard
Monitoring has become a hard problem to solve as our architectures have evolved towards Cloud, Containers and Microservices. The number of metrics seems to increase exponentially
every year as more and more company’s move towards these architectures. The single-node solutions we used to rely on (i.e. Nagios or Graphite) just don’t scale
anymore, at least not without a significant amount of time and effort, effecitvely requiring full time resources to build and manage at scale.
This growth in metrics is often referred to as cardinality. Every unique set of metric name and metric labels represents a single metric. Whereas in 2013 our largest customers
would send us 100,000 metrics (which was considered a lot at the time), nowadays its not uncommon for some of our customers to send us millions of metrics every minute.
With every new service, every new node or container, the number of metrics grows and your solution needs to scale too. For
example Kube-State-Metrics for Kubernetes can easily generate over 100,000 metrics on a single cluster,
and many organisations have multiple clusters.
At this volume, an entirely new architecture is needed, not just in the way you store and query your metrics, but also single-node architectures don’t cut it anymore. If you want to scale,
stay reliable and maintain performance, you end up having to move to a far more complex multi-node architecture to keep up with this volume of metrics.
In addition you have to meet the expectations of your users, primarily developers, who having grown used to instant search results on Google, also expect to be able to send
as many metrics as they want and be able to query them in sub-second response times. As Amy Nguyen from the Pintrest observability team has mentioned in her excellent Monitoring Design talk, “Do whatever it takes to make it fast”.
To make the problem even harder, as our environments become even more ephemeral with containers, your solution has to handle a problem called Series Churn.
For example, if each container generates 50 metrics to monitor its performance (i.e. CPU, memory etc.), that container ID label on your metrics will change everytime
your containers start and stop, adding another 50 metrics to your index each time, that only existed for the time that container was up and running. If you don’t design for this
you will see your metrics index grow extremely large very fast as we did on DalmatinerDB.
There are some properties of time-series data, especially for our use-case, IT monitoring, which can be leveraged to make the problem
far simpler to solve:
Most Data Is Only Queried In The Last Hour: We did an analysis of our queries back in 2016 and found that the value of data decreases exponentially
after only just one hour, with over 90% of the queries only looking at the last hour of data. This means you can focus most of your efforts on solving for the
newest data, and put anything older onto cheaper, albeit slower, storage and architectures.
Datapoints Are Immutable: Most datapoints come into your monitoring system in order, or at least can be organised into order over a short period of time, and
once written don’t need to be touched again. This means you don’t need to build expensive ACID database transactions,
or worry about inserts and updates once the data is written making the way you store the data much simpler.
You Can Lose a Few Datapoints Here and There: At high volume, losing a handful of datapoints is just not noticable on graphs and alerts. While you want to ensure
you store as many of the datapoints as possible, designing a system that guarentees every datapoint must be written, is a much harder problem to solve. Accepting that
its OK to lose some datapoints in extreme scenarios simplifies the problem without affecting the end user’s monitoring in any noticiable way.
Most Datapoints Don’t Change Much Between Readings: Facebook wrote a paper on this for their own TSBD, Gorrilla DB.
As many datapoints will be the same as the last reading, or only change slightly, this lends itself well to compression,
which allows you to shrink your storage resources substantially.
These challenges as well as the proprties of time-series data discussed, lead us to create a brand new architecture for our TSDB that became the
backbone of our new version of Outlyer we recently released.
Version 3: Our New TSDB Architecture
When we made the decision to move off DalmatinerDB in 2017, and redesign our architecture from the ground up to ensure we didn’t hit the same scaling and performance
issues again, we made two key decisions that helped us design the new TSDB architecture:
Whatever technologies we used had to have been proven at a larger scale than our current needs. This would mean we could scale up without being the
first person to hit technical issues with the technology and having to solve them ourselves as other people had already lead the charge and solved those
As a SaaS solution, we could build a more distributed architecture than if we had to provide the solution on-premise and worry about making it easy to install.
What this ultimately means, is we can separate the concerns of the TSDB into seperate microservices and processes that we could scale and evolve independently.
This is in contrast to a monolithic solution which may have a low setup cost, but quickly becomes a full time job to operate and manage at scale. While DalmatinerDB
did separate key parts into different services, we wanted to take this further so we could switch and replace components with higher granularity on the fly as our scale grew.
We performed another evaluation of all the TSDBs out there, looking again at InfluxDB, but also newer solutions that had appeared since 2015 such as Beringei (Gorrilla DB)
from Facebook and Atlas from Netflix.
During this process, now having a very skeptial view of amazing claims that each TSDB made when we were researching, we saw there is no “perfect” TSDB. Each solution
has its pros and cons, the dream of having a solution that you can just write metrics into and scale up linearly is just that, a dream. That’s why everyone seems to
write and release a new TSDB every week, because you can find issues with everything out there.
Using our key decision that we wanted something that was battle tested in production at significant scale already, and our
relationship with Roy Rapoport, who used to head up monitoring at Netflix and had spoken at our DOXSFO meetup last year, we
ultimately decided to go with Atlas, which is written in Scala. What we liked about Atlas, was Netflix has been using it for several
years to collect and query billions of metrics a day, it had sensible limits built in, and was in-memory so performed
super fast when we tested it.
We have learned that Atlas by itself is a component, not the full solution. It doesn’t try to do everything like shared state and replication;
every Atlas node is independent. This solves a lot of complexity and performance issues as the volume of metrics grows, but requires a lot more
tooling to manage Atlas at scale. Because we store 13 months of metrics for customers, as opposed to only 4 weeks like Netflix, putting this
all in memory for the full period would have been cost prohibitive so we had to add disk based storage to our long term nodes, which was fine
as most of the metrics on these nodes would only be queried from time to time.
Understanding this, and taking our lessons learned from Version 2, we were able to design a new architecture that has been running in production for V2
since January. The key design concepts are:
Multi-Tier Architecture: Knowing that the most recent data is the most frequently queried, we decided to split our architecture into seperate
tiers based on the retention period of the data. The most recent data is stored on in-memory nodes and loaded with new datapoints in real time, so they
can be queried instantly for the majority of users, while older data is stored on cheaper, slightly slower disk based nodes. Having multiple tiers also
solves the issue of Series Churn, as old series get removed from the short term tier within a few hours, stopping the index from growing infinitely.
Backup to S3: In parallel, the raw metrics are also written to Amazon S3, which uses a Spark along with scheduled processes in Kubernetes to perform
rollups for the longer term tiers. Using durable and reliable S3 storage as an immutable source of truth separates the concerns of short and and long term
retention. This means if we have issues on any tier, other tiers will continue working unaffected. In addition we have a full backup of all the original
data which we can use to regenerate tiers and reload instances from scratch in minutes if we spot an issue with one of our teirs. We can also use
the raw data to run batch reports for users about their environments later on.
Sharding Everywhere: We built dynamic sharding into the architecture using Consul from day one. This means our larger customers can be put onto
seperate shards, which we can independently scale and tune to their needs, and all our smaller customers can be put on a single shard so we can support them
cost effectively too. As customers grow its really easy to move them to their own shard without them noticing anything, and because each shard is dynamically
configurable via Consul, we can also provide customers with “personalised” retention settings, i.e. store metrics at any resolution they want, for however long
they want so they can control their costs on Outlyer too.
Mirrors Vs. Clustering: Clustering adds a lot of complexity and overhead to solutions as we saw with DalmatinerDB. Eventually your performance bottleneck
will be the amount of chatter between nodes trying to sync data across your network. Therefore every Atlas instance in our short term tier is an independent mirror
across availability zones. This means we have at least 2 nodes with the same data for each shard, but if one goes down, the other node can handle the load while
we automatically recreate and reload the data into the other node without users noticing. We’ve built all the tooling to do this automaticaly using
Auto-Scaling groups, which means when we actually lose a node, the system recovers automatically without any service interruption or one of our team having to
wake up to fix it.
As you can see below, the architecture uses Kafka and multiple microservices to process and query metrics, we’ve essentially unbundled the
TSBD into granular services we can independently scale and evolve. The other service on the diagram called Blend is our query engine, that handles queries from our API,
and using Consul knows to which nodes/tier portions of the query should be delegated. Blend then combines the responses into a single results which the user
sees on a graph.
Note the difference between this approach and that of DalmatinerDB, where raw data was pulled from the storage nodes and aggregated later. Most of
the work is delegated to the nodes, close to the data, with the pre-aggregated results combined by Blend.
While all the detail behind how this architecture works will be left for a seperate blog by our core TSDB engineers Dave & Chris, the results so
far have been fantastic. We’re scaling while loading queries in sub-second response times, with customers doing millions of metrics per shard. In addition
we haven’t had any fires yet, and in fact the architecture has so much resiliancy built in, it auto-heals itself for the few issues we have had without
any of our team having to wake up and fix things.
Hopefully our story will make you think twice before trying to build your own TSDB in house using open-source solutions, or if you’re really crazy, building
a TSDB from scratch. Building and maintaining a TSDB is a full time job, and we have dedicated expert engineers who are constantly improving and maintaing our TSDB,
and no doubt will iterate the architecture again over time as we hit an even higher magnitude of scale down the line.
Given our experience in this complex space, I would sincerely recommend you don’t try and do this at home, and if you have the money you should definitely outsource this
to the experts who do this as a full time job, whether its Outlyer or another managed TSDB solution out there. As so many things turn out in computing, it’s
harder than it looks!
One of the advantages we also had in Version 3, is the immense help and support from the Netflix monitoring team, in particular Brian Harrington who is the
core author of Atlas and has got on calls with us several times to help us build a successful monitoring platform around Atlas. It really does take a strong
team of super talented experts to build a TSDB that performs at scale, and its been awesome collaborating with Brian and the rest of the Netflix monitoring
Perhaps the best thing about it the book is that I release it for free. I think everyone should have the chance to learn about GraphQL, as it will change how we think about client-server communication and data management in applications, so grab your copy of it as long as it is available for free. The second best thing about the book is that I self-published this book, which means it will be updated as often as possible, and that I can keep the PDF, EPUB, or MOBI versions in a state of constant improvement.
Hexbyte Hacker News Computers The Road to GraphQL
I wrote the The Road to GraphQL over the last year, while building several GraphQL applications for my clients and for myself. During this time, I came to understand the practical genius of GraphQL, and how it dramatically improves communication in client-server architectures. Not only does it improve the interface between the client and the server, it also enhances client-side state management by eliminating remote data management. Sophisticated GraphQL libraries like Apollo Client provide powerful features like caching, optimistic UI, and data prefetching for free. This book covers the fundamentals of GraphQL itself, as well as its ecosystem.
The book comes with 350 pages, 45+ exercises, 400+ code snippets, 4 applications you are going to build along the way and various source code applications if you decide to buy the complete course. I applied the same principles as my other books:
Keep it simple
Answer the why, not just the how
Experience a problem, solve a problem
Client-side GraphQL with React
The first part teaches how to build a GraphQL client application in React. Understanding React is a requirement before this book, so I recommend you take a look at my other free book called “The Road to learn React” if you need a good course. At some point, you will see the tedium in performing every GraphQL operation with basic HTTP requests. That’s when we’ll swap basic HTTP implementation for Apollo Client. While building your application with Apollo Client, you will learn more advanced features like caching, optimistic UI, and pagination with Apollo in React. In the end, you will have created a functional GraphQL client application.
Server-side GraphQL with Node
Table of Contents
What is GraphQL?
Why not REST?
GraphQL Setup, Tools and APIs
Feeding the API with Data on GitHub
Read/Write Data with GitHub’s Personal Access Token
Interacting with GitHub’s GraphQL API
GraphQL Operation: Query
GraphQL Operation: Mutation
React with GraphQL
Writing your first React GraphQL Client
GraphQL Query in React
GraphQL Pagination in React
GraphQL Mutation in React
Shortcomings of GraphQL in React without Apollo
Starting with Apollo Boost on the Command Line
Apollo Client and a GraphQL Query
Apollo Client with Pagination, Variables, Nested Objects and List Fields
Apollo Client and a GraphQL Mutation
React with GraphQL and Apollo Client
Writing your first React with GraphQL and Apollo Client
Configure Apollo Client for React and GitHub’s GraphQL API
Connect Data-Layer to View-Layer: Introducing React Apollo
GraphQL Query with Apollo Client in React
Apollo Client Error Handling in React
GraphQL Mutation with Apollo Client in React
GraphQL Query/Mutation with Higher-Order Components in React
Local State Management with Apollo Client in React
Apollo Client Optimistic UI in React
GraphQL Pagination with Apollo Client
GraphQL Caching of Queries with Apollo Client
Implementing the Issues Feature: Setup
Implementing the Issues Feature: Client-Side Filter
Implementing the Issues Feature: Server-Side Filter
Apollo Client Prefetching
Exercise: Commenting Feature
Appendix: CSS Files and Styles
Node.js with GraphQL and Apollo Server
Apollo Server Setup with Express
Apollo Server: Type Definitions
Apollo Server: Resolvers
Apollo Server: Type Relationships
Apollo Server: Queries and Mutations
GraphQL Schema Stitching with Apollo Server
PostgreSQL with Sequelize for a GraphQL Server
Connecting Resolvers and Database
Apollo Server: Validation and Errors
Apollo Server: Authentication
Authorization with GraphQL and Apollo Server
GraphQL Custom Scalars in Apollo Server
Pagination in GraphQL with Apollo Server
Testing a GraphQL Server
Batching and Caching in GraphQL with Data Loader
GraphQL Server + PostgreSQL Deployment to Heroku
I look forward to extending this book with more technologies and features from the GraphQL specification. I hope “The Road to GraphQL” captures my enthusiasm for the topic, and that it becomes a valuable resource among other courses, tutorials, and books on pragmatic GraphQL programming. As always, feedback through any of the available platforms is welcome and encouraged!
Hexbyte Hacker News Computers The Road to learn React
Build a Hacker News App along the way. No setup configuration. No tooling. No Redux. Plain React in 190+ pages of learning material. Learn React like 33.000+ readers.