Presentations

9 biases in tech a talk about different biases that skew the technical decision process we deal with in our day-to-day operations. every day we make decisions. those decisions can be around improving your product, process, company; sometimes even bettering yourself. but many of our decisions are not made consciously. we all have biases. and even though most of our biases are unconscious, those biases still can get in a way of making the best possible decision. this is not a talk about gender and racial bias in tech hiring, this is a talk about different biases that skew the technical decision process we deal with in our day-to-day operations. robert treat

securing your data on postgresql a look at the many options postgres provides for securing your data this talk will cover the various features that postgres provides for data security, from the very basic to the most advanced. while most applications are aware of the minimum basic security features and use them, there is often a lack of understanding about how best to manage them, especially with major security features being released with every major version of postgres. as for advanced features, sadly most of them go unnoticed and unused in most cases. the topics that will be covered are: - host based authentication in postgres, types of authentications available, and how this differs from the way other rdbms manage users and authentications, especially mysql. - peculiarities of postgres in permissions and acl. - setting up and using ssl/tls for connections and certificate management - row-level security - event triggers - implementing pci security standards for storing credit card data. - using appropriate filesystem permissions - encrypting your data stored in postgres - implementing table level auditing with minimal storage requirements within the database, and other alternatives for auditing. - reviewing and rejecting sql injections - other postgresql security features - tips to leverage additional tools in the cloud if you are using postgres as a service (e.g. postgresql rds by aws) - upcoming security features in postgres 11 - features that postgres currently lacks, and gotchas in existing features. the speaker will also discuss recommended monitoring to ensure security implementations set up are working as intended, especially with pci. the primary intent of this talk is to spread information about security features in postgres, and the right way to implement them. there seems to be a lot of comparisons between security features of various rdbms, but few would disagree with the view that the best way to decide the right one to use is by thorough knowledge of what each provides. and this talk aims to do just that for postgres and data security. payal singh

pandas - lazy at mating, lively at mining kick start your data mining activities without having to invest time learning complicated tools. data mining is a unique blend of hardcore math and fine art. it includes various disciplines such as machine learning, data visualization, critical analysis, common sense, logical thinking, differential calculus, linear algebra, matrices and sets, and many more. it is easy to feel intimidated at the onset of becoming a miner, which is where this session comes in to lay out in very simple terms how exciting and beautiful the world of data mining is, and how anyone with a love for discovering hidden information in data can use python, pandas, scikit and pyplot to find hidden truths. aside from demonstrating how quick, easy and painless mining can be if approached correctly with the right tools, the talk will also aim to give a more general perspective of discovering truths in most optimal ways, and blowing the audience's minds with relating the techinues of discovering relationships in data through data mining with general relativity and wormholes. the intent of this presentation is help people kick start their data mining activities without having to invest time learning complicated tools, and to get people excited about getting their hands dirty with data. "stuff your eyes with wonder," he said, "live as if you'd drop dead in ten seconds. see the world. it's more fantastic than any dream made or paid for in factories." - ray bradbury (fahrenheit 451) payal singh

postgresql partitioning - the new legacy covers postgres partitioning including changes in postgres 10 and 11 postgresql table partitioning allows you to scale your data to significantly larger volumes than would otherwise be possible. getting started with table partitions in and of itself is not complicated, but if you want to get maximum benefits from your setup you can't just follow the documentation. this talk will walk through the ins and outs of table partitioning including: what table partitioning is, and how it is implemented in postgresql. inheritance, constraints, triggers, and how it all comes together. partition maintenance. learn how to setup and maintain table partitioning in postgresql with hands on examples. we'll walk through manipulating sql, writing triggers, and how you can automate your partition maintenance. robert treat

developing applications for performance delivered at codercruise scalability != performance. in fact, having to scale your architecture significantly with growth may be a symptom of a poor application performance. yet, with the rise of cloud and the abundance of automation and container tools that simplify scalability aspect of your system, performance considerations are often pushed to the back row. building systems for high performance is not easy. it requires a lot of considerations - from technology selection to design decisions. and "the cloud" does not magically solve those problems for you. in this talk i'll discuss common performance pitfalls across the stack and talk about useful techniques and examples that every application could benefit from. leon fayer

securing your data on postgres securing your data on postgres while most applications are aware of the minimum basic security features and use them, there is often a lack of understanding about how best to manage them, especially with major security features being released with every major version of postgres. as for advanced features, sadly most of them go unnoticed and unused in most cases. i will also discuss recommended monitoring to ensure security implementations set up are working as intended, especially with pci. payal singh

production testing through monitoring given at devopsdays salt lake city 2017 this talk covers inefficiencies of some common testing methods, provide real world examples of discovering edge cases with monitoring, and offer recommendations on embracing devops culture to increase observability and help with identifying business-effecting problems. robert treat

building the right architecture for you delivered at devopsdays austin 2017 <p> in the world of agile methodologies, minimal viable product (mvp) is a widely accepted concept among application developers. mvp is geared towards developing just enough features to gather validated learning about the product and its continued development. not only this approach improves time to market, but it is also significantly cheaper from the perspective of both the upfront investment and cost of change to the direction of the product based on feedback. however, that same mentality haven't really crossed into the realm of underlying architecture. </p> <p> as an industry, we're fascinated with solutions that are presented and blogged about by companies working with the bleeding edge technologies. problem is - most of us don't have the problems those companies are trying to solve. but many don't try to find a solution that matches the problem at hand. that's why reference architectures are treated as tutorials instead of references and technologies are picked based on their rankings on reddit. in this talk i discuss some of the anti-patterns for technology selection and architecture design as well as talk about the approach to designing a minimum viable architecture for your use case. </p> leon fayer

managing os provided postgresql packages managing os provided postgresql packages <p>keith fiske presents: <a href="http://slides.keithf4.com/managing_pg_packages/#/">managing os provided postgresql packages</a></p> keith fiske

provisioning and automating high availability postgres on aws ec2 presented at pgconf us 2017 rds is great. but sometimes you feel the need for more flexibility and customization that is just not possible with rds. in this talk i will talk about why one might want to go with setting up and maintaining a ha postgres cluster on ec2 instead of rds, following which i will go through each of the steps for setting up such a system and automating installation and upgrades, replication, backups and restores with s3, pitr, monitoring with cloudwatch and failover. this talk is especially useful for people looking to gain a better understanding of how the various aws tools can be used together for a ha postgres cluster, and as a howto guide on migrating from either rds or their own hosted cluster on to ec2. payal singh

lost art of troubleshooting delivered at devopsdays baltimore 2017 <p> there are a lot of great things about the cloud, but the "destroy and rebuild" philosophy which is really good for building a continuous delivery pipeline, really sucks when applied to troubleshooting production problems. when your application goes haywire, the most valuable engineering skill is not the the ability to bring up a copy of your system or even the knowledge of your technology stack (although it doesn't hurt). it is the skill of understanding and solving problems. </p> <p> finding the root cause of the issue and mitigating it with minimal disruption in production is a must-have skill for engineers responsible for managing and maintaining production systems, which nowadays includes ops, dbas and devs alike. in this talk i will discuss the skills required to troubleshoot complex systems, traits that prevent engineers from being successful at troubleshooting and discuss some techniques and tips and trick for troubleshooting complex systems in production. </p> leon fayer

bizops and you delivered at devopsdays boston 2016 there are plenty of materials on getting development and operations to work together. more conversations are happening around inclusion of other technology groups, such as dbas and qa testers, into devops processes. that said, devops conversations has been largely devoid of talk about bizops place at the table. the goal for any tech-centric group is not to build and/or architect the best technology, but rather to effectively support business. yet, many of those groups are either not privy to or don't bother understanding the business goals and overarching effects of the technical decisions made. in this talk i'll discuss key areas and feedback points in every devops process fit for inclusion of business units in order to align technology and business goals and make your life easier. leon fayer

managing databases in a devops environment delivered at pgdayphilly 2016, this talk covers how configuration management, monitoring, and rapid deployments are impacting how we think about database management. delivered at pgdayphilly 2016, this talk covers how configuration management, monitoring, and rapid deployments are impacting how we think about database management. robert treat

oncall for developers presented at devopsdays atlanta 2016 with emergence of devops approach to application development, deployment and management developers get more and more involved in day-to-day system operations. lately, there has been a popular point of view that developers should be included in oncall rotation on equal grounds with sys admins. while i don't fully subscribe to that mentality, there are certain processes that must be implemented by every organization to get developers involved in production operation of the software they built. in this talk i'll walk through different aspects of operational oncall responsibilities and discuss ways in which developers should (and should not) be involved in operation of production systems. leon fayer

database performance 101 presented at dc-baltimore perl workshop 2016 being involved in performance audits on systems of every size, from start-up sites hacked together overnight, to a ginormous applications built by world-recognized brand companies, i’ve seen a lot of interesting (and sometimes very unique) performance issues in every level of the stack: code, architecture, databases (sometimes all of the above). but there are a few particular, very “performance 101″, issues that (unfortunately) appear in a lot of code bases. in this talk i'll present the most common database-related performance bottlenecks that can happen in most applications. leon fayer

less alarming alerts - srecon 2016 presented at srecon 2016 alert overload is bad for people and bad for business. in this presentation we discuss some of the techniques we have learned in our operations practice for getting business on board with actionable alerts, and how to prune poor alerts from your system. robert treat

sandboxing openzfs on linux presented at openzfs developer summit <p> on illumos, *zones* are sandboxed environments encompassing filesystem, network, ipc and other resources, as well as fine-grained resource controls. zones are backed by a persistent configuration store. all system calls can take advantage of zone awareness to isolate resources and process privileges. a list of zfs datasets can be *delegated* to a zone, which makes them visible to processes inside the zone and allows administrative operations on the datasets and their children. </p>​ <p> the linux kernel is different in that there is no first-order object corresponding a specific virtual environment. instead, linux provides a set of primitives for resource isolation and control. each process is associated with a list of *namespaces*, each of which isolates a specific type of resource, such as mountpoints and network interfaces. processes are also bound to *cgroups*, or control groups, which form a heirarchy for each available resource control. native *container* frameworks on linux work by composing sets of namespaces and cgroups and creating processes using them. </p> <p> we are building a container platform based on linux and openzfs, and one functionality gap we're addressing is to provide zfs delegation to processes running inside linux containers. we introduce a *dataset namespace*, which functions as an analogue of a zone identifier. in the solaris porting layer (spl) portion of zfs on linux, we added zone interfaces and associate zone objects with dataset namespaces. this allows many of the existing zone paths in zfs to be reused. since the namespace infrastructure has no persistence, new entry points have to be added to manage the list of datasets when a new namespace is created. </p> albert lee

production testing through monitoring presented at devopsdays charlotte identifying and fixing issues in new code before deploying it to production is important for every software development cycle. however, relying on traditional testing methods in the age of internet-scale data driven problems may prove to be incomplete. identifying and fixing the issues in production quickly is crucial, but it requires insight into usage patterns and trends across the whole architecture and application logic. in this talk i will touch on inefficiencies of some of the most common testing methods, provide real world examples of discovering odd edge cases with monitoring and offer recommendations on top-down metric instrumentation to help devops organizations with identifying and acting on business-effecting problems. leon fayer

postgres 9.4 replication tutorial a tutorial for setting up postgres replication <p>most people use at least one 3rd party script/tool to manage postgres replication for building reliable solution for mission critical postgres databases. postgres 9.4's new features allow users to setup replication using built in features without compromising reliability and remove reliance on 3rd party tools.</p> <p>in this tutorial, i will walk you through replication options available in postgres 9.4, selecting the right solution to achieve your business requirements, and things to consider for monitoring and maintaining your setup in production. this is a hands on tutorial so please bring your laptop with you. i will share vms in advance that will be used during tutorial to setup replication with postgres 9.4.</p> <p>training vm:</p> <p>the postgres training vm can be downloaded from the one of the links below. the zip file is about 2gb and it unzips to about 6gb, so people should make sure they have enough space free for it. i'll be bringing it along on a usb stick as well in case everyone doesn't have a chance to grab it.</p> <p>download vm from any of the following links:</p> <p>(1) see training vm download link under links on this page</p> <p>(2) google drive: https://drive.google.com/open?id=0bxnxwkt5prbdevbyqvyyskhiemc</p> <p>after downloading above file, it should be uncompressed. to bring up the vm, you should virtualbox software installed on the machine. an existing vm can be pulled into virtualbox by going to the menu "machine -> add.." and then browsing to wherever the vm was extracted to.</p> <p>i highly encourage to download the vm before the training so we can use time to cover postgres replication topic in detail.</p> <p>let me know via denish@omniti.com , if you see any problem downloading it.</p> denish patel

keeping your product owner productive presented at devopsdays austin explores how devops and the business can get along better in agile shops. clinton wolfe

what devops is not presented at devopsdays dc 2015 devops has been a hot topic in the industry for some time now. a lot of people been talking about it. some have built business models around devops-related tools and themes. there are even conferences and trade shows dedicated to devops-oriented things. people have made career around talking about it. in light of all of that, i find it chuckle-worthy that very few people actually know what devops is. so instead of trying to create a buzzword-infested definition of devops to suit my particular agenda, i’d like to talk about what devops is not. leon fayer

your goat antifragiled my snowflake!: demystifying devops jargon presented at chefconf 2015 are you a cow, a pet, a canary, or a unicorn? do you prefer blue/green, or red/green/refactor? who the heck is brent? welcome to devops, where we are all about breaking down walls. but, we've created a private dialect, full of familiar words with unfamiliar meanings, and in-jokes upon in-jokes. many newcomers wish there was a glossary for the movement. time to be inclusive! clinton wolfe

think your postgres backups and disaster recovery are safe? let's talk. presented at pgconfnyc 2015 disaster recovery is perhaps the most ignored aspect of managing postgresql in production systems, and its value is only realized in a moment of crisis. if not done right, absence of backup and recovery strategies can make one repent for days, if not months after. even when catered to, majority of production systems only have at most one defined backup and recovery strategy, not realizing the benefits of using a hybrid solution or a combination of multiple types of backups and recovery approaches. contrary to common perception of treating disaster management as a one model fits all philosophy, emphasis should be given on how various types of backups have been optimized for specific kinds of disaster scenarios. payal singh

when postgresql can't, you can! presented at pgday.cz we love using postgresql, but that doesn't mean there aren't a few things missing that would make using it in large production environments easier: automated partitioning, autonomous functions, logical replication & better dump filtering. fortunately, it's extremely easy to make add-ons to allow some of those features right now. keith fiske

don't forget the elephant delivered at the central pa open source conference 2014 <p>many database administrators were first introduced to postgresql years ago and have memories (good & bad) of using version numbers starting with 6, 7 & 8. many people are still running these old versions as well, nervous about upgrading and unsure of the benefits. with the rise in popularity of mysql, many open source advocates turned there since the majority of internet community support was centered around it. and the advanced features that commercial databases such as oracle & sql server were advertising appealed to many enterprise administrators.</p> <br /> <p>but in the last several years, things have changed. mysql is still one of the most popular open source databases, but its ownership by oracle has cast a vast shadow over the project. and the rising costs and license management of commercial database systems are causing enterprise headaches. nosql has also taken off as an alternative data storage model, but the implementations have often been left lacking long term, essential data management features. in all this time, postgresql has been getting many of those advanced features it used to be missing and has been getting some of its own unique capabilities that can set it above the rest.</p> <br /> <p>this talk will go over the major feature improvements in postgresql since the 8.x series was released, how its community has stepped up to fill in the missing feature gaps, and what the future holds for one of the longest running, truly open source relational database management system projects.</p> keith fiske

what ops can learn from design delivered at velocity europe in barcelona <p class="publongdes">delivered at velocity europe in barcelona, this talk introduces "ops" people to the idea of user centered design, touching on several techniques long used in the design world, and talks about how those ideas might be applied to software and processes that we use every day.</p> robert treat

php performance 101: so you need to use a database presented at php world, november 2014 <p class="publongdes">being involved in performance audits on systems of every size, from start-up sites hacked together overnight, to a ginormous applications built by world-recognized brand companies, i’ve seen a lot of interesting (and sometimes very unique) performance issues in every level of the stack: code, architecture, databases (sometimes all of the above). but there are a few particular, very “performance 101″, issues that (unfortunately) appear in a lot of code bases. in this talk i present the most common database-related performance bottlenecks that can happen in most php applications.</p> leon fayer

improving devops through better monitoring presented at devopssummit, november 2014 <p class="publongdes">some developers believe that monitoring is a function of operations team. some operations teams firmly believe that monitoring the systems they maintain is sufficient to run the business successfully. most of them are wrong. the complexity of today’s applications have gone far and beyond the capabilities of “traditional” system-level monitoring tools and approaches and requires much broader knowledge of business and application as a whole. the goal of devops is to connect all aspects of application development and operations, and monitoring provides visibility and troubleshooting tools to accomplish that goal.</p> <p class="publongdes">this talk is intended to provide real-world examples of common gaps in monitoring approach and explain why holistic instrumentation of business and functionality monitors should be a part of any project scope.</p> leon fayer

postgres in amazon rds presented at pgcon 2014 <p class="publongdes">in recent years, amazon’s cloud service has become key for deploying and scaling it infrastructure tiers, and this is no different for postgres. amazon’s relational database service for postgres rds is a new, but compelling option for startups, when resources are scarce and administrators are hard to find! rds is a great option for people in terms of rapid deployment of replicas, ease of failover and the ability to easily redeploy hosts when failures occur, rather than spending extensive time trying to repair. this talk covers topics related to managing postgres rds.</p> denish patel

achieving pci compliance with postgres presented at pgconf 2014 <p class="publongdes">in this talk, denish discusses guidelines and requirements from pci dss v 3.0 and how postgres powerful security features help to address pci compliance requirements.</p> denish patel

writing maintainable perl presented at dcbpw, may 2014 <p class="publongdes">one criticism of opponents of perl is that it is a "write-only" language meaning that once the code is written, it is extremely difficult to maintain because it is difficult to understand upon re-examination. as with many criticisms, this should be aimed at those undisciplined developers who are writing the code, and not their tool of choice.</p> <p class="publongdes">having said that, i think it is also fair to say that perl makes it very easy to write difficult-to-decipher code. this is the double­-edged sword which is the shorthand perl gives us to be very expressive in a small amount of space. a negative application of this is obfuscated perl (where the author intentionally makes his code difficult to read), while a more positive application is the craft of creating perl "one­liners" (trying to include a great deal of functionality in a single line of code). a one­liner can be a powerful weapon in the arsenal of a system administrator.</p> david bradford

a scalability call to action plenary session: surge 2010 <p class="publongdes">10 commandments (+1) of scale direct from theo schlossnagle at the inaugural surge conference.</p> theo schlossnagle

managing databases in a {devops} environment presented at pgopen 2011 and velocity 2011 <p class="publongdes">there’s a lot of talk in the devops world about bringing developer concepts to system administration, and discussion the other way about bringing the awareness of operations to developers, but a lot of the conversation leaves out what is often the most critical part of your technology stack: the database.</p> robert treat

the essential postgresql.conf presented at jdcon east 2011 <p class="publongdes">this talk cover the most important postgresql.conf variables that users should tune and/or study up on. it is geared toward postgres 9.0.x servers.</p> robert treat

monitoring and observability in complex architectures what needs monitoring and why <p class="publongdes">a whirlwind tour through the complexity of modern internet architectures. the complexities lead to evil optimization problems, and significant challenges troubleshooting production issues to a speedy and successful end.</p> theo schlossnagle

less alarming alerts presented at o'reilly velocity europe 2013 <p class="publongdes">at omniti, we’re often forced to walk into the middle of an existing infrastructure that is already set on fire. the only thing worse than having no alerts in that situation is having hundreds of alerts screaming at you constantly. over the years we’ve had to come up with a way to help keep our operations team sane while also providing business value, and most importantly giving comfort to the folks that have brought us in. the methods that we’ve developed can be used by any operations team to help bring sanity back to their world, and end the cycle of “pager fatigue”.</p> robert treat

advanced wal file management with omnipitr presented at pgeast 2011 <p class="publongdes">files from talk on omnipitr, delivered at pgeast 2011 in nyc.</p> robert treat

when postgresql can't, you can filling the gaps <p class="publongdes">after you've been using postgresql for a while, you realize that there are some things still missing that would make using it in large, production environments easier: automated partitioning, autonomous functions, logical replication & better dump filtering. fortunately, it's extremely easy to make add-ons to allow some of those features right now.</p> keith fiske

past, present and pachyderm presented at all things open 2013 <p class="publongdes">slides based on my talk at the all things open conference, held in raleigh, north carolina. this talk covers some basic history on postgres, new features in the 9.3 release, and some thoughts on what might be in the future for postgres.</p> robert treat

building scalable systems: an asynchronous approach asynchronous approach to building scalable systems <p class="publongdes">a brief tour through asynchronous systems.</p> theo schlossnagle

pro postgres 9 presented at jdcon east 2011 <p class="publongdes">this talk gives an overview of the postgres community, the postgres database, and several add-on tools for postgres. it is intended to provide a guide for someone wanting to do full time postgres administration.</p> robert treat

deploying maximum ha architecture with postgresql deploying maximum ha architecture with postgresql <p class="publongdes">why do you need ha architecture? why postgresql? goals for maximum ha.</p> denish patel

disposable environments at scale presented at zfs day 2012 <p class="publongdes">how i learned to stop worrying and love zfs</p> eric sproul

scaling postgres scaling postgres <p class="publongdes">options available in postgres</p> denish patel

omnios motivation and design building an illumos-based os <p class="publongdes">we needed a path of least resistance for our solaris 10 users that delivered on the promises sun made.</p> theo schlossnagle

big, bad, "upgraded" postgres what is big, bad upgraded postgres? <p class="publongdes">this talk covers a long running upgrade project of a multi-terabyte database from postgres 8.3 to postgres 9.1, by way of pg_migrator. both technical and non-technical reasons why the project took several years to complete is discussed.</p> robert treat

high-performance robust http front-ends presented at rit++ <p class="publongdes">the http caching talk given at rit++ 2011</p> theo schlossnagle

monitoring is easy: why do we suck at it? monitoring presentation null theo schlossnagle

dashboards for web operations presented at velocity 2011 <p class="publongdes">what you should think about putting in webops dashboards.</p> theo schlossnagle

postgresql on solaris postgresql on solaris null theo schlossnagle

database anti-patterns database anti-patterns <p class="publongdes">presented at pgcon 2008.</p> robert treat

scalable internet architectures scalable internet architectures null theo schlossnagle