Are you over 18 and want to see adult content?
More Annotations
A complete backup of computersharevoucherservices.com
Are you over 18 and want to see adult content?
A complete backup of yohgen-music.com
Are you over 18 and want to see adult content?
A complete backup of usnewsuniversitydirectory.com
Are you over 18 and want to see adult content?
A complete backup of stephencurryshoes.biz
Are you over 18 and want to see adult content?
Favourite Annotations
A complete backup of ajarproductions.com
Are you over 18 and want to see adult content?
A complete backup of desafiosesucesso.com.br
Are you over 18 and want to see adult content?
A complete backup of codingmonkeys.de
Are you over 18 and want to see adult content?
A complete backup of readonlymemory.vg
Are you over 18 and want to see adult content?
A complete backup of iskanevleriasm.com
Are you over 18 and want to see adult content?
Text
THE HFT GUY
My Experience In Production with: Flask, Bottle, Tornado and Twisted. 27 October 2020. 19 October 2020 thehftguy. 6 Comments. i. 9 Votes. I’ve been using flask, bottle, tornado in production day to day for years, writing plugins for these and maintaining a variety of new or legacy applications that happened in Big Corp TM. FRENCH JUDGE RULES GPL LICENSE TO BE INAPPLICABLE INSEE MORE ONTHEHFTGUY.COM
A HAVEN FOR FRAUD AND STOLEN CARDS… OR HOW MY WIFE WASSEE MORE ONTHEHFTGUY.COM
WHAT AES CIPHERS TO USE BETWEEN CBC, GCM, CCM, CHACHA-POLYSEE MORE ONTHEHFTGUY.COM
THE DIFFERENCES BETWEEN SPLUNK, KIBANA AND GRAYLOG. Choice is open between: Splunk, a tool with incredible joining and charting capability, that is dog slow and cost a fortune. ElasticSearch (both Kibana and Graylog), a lightning fast distributed database, that can’t do a join and might crash if a message is a string or is not a string. POSTMORTEM: WHY YOU SHOULD NOT RE-USE CSR WHEN RENEWINGSEE MORE ONTHEHFTGUY.COM
MAKING BEAUTIFULSOUP PARSING 10 TIMES FASTER BeautifulSoup. 1. 2. soup = BeautifulSoup (r.content, 'http.parser') Translation: 4.28 seconds to download 4 pages ( requests.api + requests.sessions) 7.92 seconds to parse 4 pages ( bs4.__init__) The HTML parsing is extremely slow indeed. Looks like it’s spending 7 seconds just to detect the character set of the document. CONFIGURING TIMEOUTS IN HAPROXY THE DIFFERENCE BETWEEN ID_TOKEN AND ACCESS_TOKEN IN OPENID As a developer, you need the user identifier. Typically a primary key, an email or an employee ID. The two tokens are mostly equivalent, except for the mail field. Let’s assume that users are referenced by email in this example. The access_token is worthless here. It’smissing the email.
CRACKING THE HACKERRANK TEST: 100% SCORE MADE EASY Foreword It's well known that most programmers wannabes can't code their way out of a paper bag. As a consequence, the tech industry is pushing for longer, harder and evermore extreme screening. The whiteboard interview has been the standard for a while, followed by puzzles , then FizzBuzz. The latest fad is HackerRank.It's
THE HFT GUY
My Experience In Production with: Flask, Bottle, Tornado and Twisted. 27 October 2020. 19 October 2020 thehftguy. 6 Comments. i. 9 Votes. I’ve been using flask, bottle, tornado in production day to day for years, writing plugins for these and maintaining a variety of new or legacy applications that happened in Big Corp TM. FRENCH JUDGE RULES GPL LICENSE TO BE INAPPLICABLE INSEE MORE ONTHEHFTGUY.COM
A HAVEN FOR FRAUD AND STOLEN CARDS… OR HOW MY WIFE WASSEE MORE ONTHEHFTGUY.COM
WHAT AES CIPHERS TO USE BETWEEN CBC, GCM, CCM, CHACHA-POLYSEE MORE ONTHEHFTGUY.COM
THE DIFFERENCES BETWEEN SPLUNK, KIBANA AND GRAYLOG. Choice is open between: Splunk, a tool with incredible joining and charting capability, that is dog slow and cost a fortune. ElasticSearch (both Kibana and Graylog), a lightning fast distributed database, that can’t do a join and might crash if a message is a string or is not a string. POSTMORTEM: WHY YOU SHOULD NOT RE-USE CSR WHEN RENEWINGSEE MORE ONTHEHFTGUY.COM
MAKING BEAUTIFULSOUP PARSING 10 TIMES FASTER BeautifulSoup. 1. 2. soup = BeautifulSoup (r.content, 'http.parser') Translation: 4.28 seconds to download 4 pages ( requests.api + requests.sessions) 7.92 seconds to parse 4 pages ( bs4.__init__) The HTML parsing is extremely slow indeed. Looks like it’s spending 7 seconds just to detect the character set of the document. CONFIGURING TIMEOUTS IN HAPROXY THE DIFFERENCE BETWEEN ID_TOKEN AND ACCESS_TOKEN IN OPENID As a developer, you need the user identifier. Typically a primary key, an email or an employee ID. The two tokens are mostly equivalent, except for the mail field. Let’s assume that users are referenced by email in this example. The access_token is worthless here. It’smissing the email.
CRACKING THE HACKERRANK TEST: 100% SCORE MADE EASY Foreword It's well known that most programmers wannabes can't code their way out of a paper bag. As a consequence, the tech industry is pushing for longer, harder and evermore extreme screening. The whiteboard interview has been the standard for a while, followed by puzzles , then FizzBuzz. The latest fad is HackerRank.It's
BORKED WINDOWS 10 UPDATE 2020-08-31: RESET SOUND SETTINGS Borked Windows 10 Update 2020-08-31: Reset Sound Settings. I’ve decided to open a recurrent series “ Borked Windows Update ” to narrate my adventures with broken Windows updates. In the 2 years I’ve been using Windows 10 (had to upgrade because Windows 7 was nearing end of life), I personally faced about 5 major-breakingupdates
TECHNICAL DEBT DOESN’T EXIST After decades of software engineering, I came to the professional conclusion that technical debt doesn't exist. Oh I have seen software rewritten countless times under the pretense of technical debt, doesn't mean there was technical debt unless "technical debt" is defined as "any existing code". Software is unique in the way thateverything that was
MAKING BEAUTIFULSOUP PARSING 10 TIMES FASTER BeautifulSoup. 1. 2. soup = BeautifulSoup (r.content, 'http.parser') Translation: 4.28 seconds to download 4 pages ( requests.api + requests.sessions) 7.92 seconds to parse 4 pages ( bs4.__init__) The HTML parsing is extremely slow indeed. Looks like it’s spending 7 seconds just to detect the character set of the document. OPENDJ – THE HFT GUY A bit of history: OpenDJ was originally developed by Sun, who invented Java and the JVM. The story is that used internally OpenDJ as a benchmark during the development of the G1 garbage collector. This is the latest java garbage collector recommended when using more than 8 WILL AIRBNB GO BANKRUPT? AND WHEN? Read the news: Airbnb Racks Up Hundreds of Millions in Losses Due to Coronavirus AirBnb is losing money at the speed or light and that will only get worse over the coming months. The question on everyone's mind is. Will AirBnb die? and more importantly, BEFORE AND AFTER DOCKER: HOW TO DEPLOY AN Docker is a packaging and deployment system. It allows you to package an application as a "docker image", then deploy it easily on some servers with a single "docker start " command. Packaging an application Packaging an application without Docker A developer pushes a change The CI sees that new code is available. It rebuilds JWT IS AWESOME: HERE’S WHY Here's my take on JWT* and why I think it's amazing, after having migrated hundreds of applications in JP Morgan from a legacy authentication solution. *JWT = JSON Web Token 1) Pro: JWT is standardized and supported in most languages For applications to have authentication, this requires developers to tightly integrate authentication when writing the 250 GB/DAY OF LOGS WITH GRAYLOG: LESSONS LEARNED Storage (as measured in production) 138 906 326 logs per day (averaged over the last 7 days) 2200 GB used, for 9 days of data. 1800 bytes/log in average. Our current logs require 250 GB of space per day. 12 TB will allow for 36 days of log history (at HOW TO PRESENT A GITHUB PROJECT FOR YOUR RESUME Introduction Companies ask for a GitHub profile. Recruiters ask for a GitHub profile. The question "Do you contribute to open-source?" is now one of the most common questions asked in phone screens. If people want a GitHub, we shall give them a GitHub. This article will explain how to present a GitHub project for use HACKERRANK TESTING: A GLIMPSE AT THE COMPANY SIDE HackerRank is an online coding platform. It provides coding tests and questions for companies to screen candidates. We remember the first time we had to do a test (before joining the company), unsure what were the expectations. Later, we were designing new tests (after joining the company), unsure what to expect from candidates. Wedecided
THE HFT GUY
My Experience In Production with: Flask, Bottle, Tornado and Twisted. 27 October 2020. 19 October 2020 thehftguy. 6 Comments. i. 9 Votes. I’ve been using flask, bottle, tornado in production day to day for years, writing plugins for these and maintaining a variety of new or legacy applications that happened in Big Corp TM. FRENCH JUDGE RULES GPL LICENSE TO BE INAPPLICABLE INSEE MORE ONTHEHFTGUY.COM
A HAVEN FOR FRAUD AND STOLEN CARDS… OR HOW MY WIFE WASSEE MORE ONTHEHFTGUY.COM
WHAT AES CIPHERS TO USE BETWEEN CBC, GCM, CCM, CHACHA-POLYSEE MORE ONTHEHFTGUY.COM
THE DIFFERENCES BETWEEN SPLUNK, KIBANA AND GRAYLOG. Choice is open between: Splunk, a tool with incredible joining and charting capability, that is dog slow and cost a fortune. ElasticSearch (both Kibana and Graylog), a lightning fast distributed database, that can’t do a join and might crash if a message is a string or is not a string. POSTMORTEM: WHY YOU SHOULD NOT RE-USE CSR WHEN RENEWINGSEE MORE ONTHEHFTGUY.COM
MAKING BEAUTIFULSOUP PARSING 10 TIMES FASTER BeautifulSoup. 1. 2. soup = BeautifulSoup (r.content, 'http.parser') Translation: 4.28 seconds to download 4 pages ( requests.api + requests.sessions) 7.92 seconds to parse 4 pages ( bs4.__init__) The HTML parsing is extremely slow indeed. Looks like it’s spending 7 seconds just to detect the character set of the document. CONFIGURING TIMEOUTS IN HAPROXY THE DIFFERENCE BETWEEN ID_TOKEN AND ACCESS_TOKEN IN OPENID As a developer, you need the user identifier. Typically a primary key, an email or an employee ID. The two tokens are mostly equivalent, except for the mail field. Let’s assume that users are referenced by email in this example. The access_token is worthless here. It’smissing the email.
CRACKING THE HACKERRANK TEST: 100% SCORE MADE EASY Foreword It's well known that most programmers wannabes can't code their way out of a paper bag. As a consequence, the tech industry is pushing for longer, harder and evermore extreme screening. The whiteboard interview has been the standard for a while, followed by puzzles , then FizzBuzz. The latest fad is HackerRank.It's
THE HFT GUY
My Experience In Production with: Flask, Bottle, Tornado and Twisted. 27 October 2020. 19 October 2020 thehftguy. 6 Comments. i. 9 Votes. I’ve been using flask, bottle, tornado in production day to day for years, writing plugins for these and maintaining a variety of new or legacy applications that happened in Big Corp TM. FRENCH JUDGE RULES GPL LICENSE TO BE INAPPLICABLE INSEE MORE ONTHEHFTGUY.COM
A HAVEN FOR FRAUD AND STOLEN CARDS… OR HOW MY WIFE WASSEE MORE ONTHEHFTGUY.COM
WHAT AES CIPHERS TO USE BETWEEN CBC, GCM, CCM, CHACHA-POLYSEE MORE ONTHEHFTGUY.COM
THE DIFFERENCES BETWEEN SPLUNK, KIBANA AND GRAYLOG. Choice is open between: Splunk, a tool with incredible joining and charting capability, that is dog slow and cost a fortune. ElasticSearch (both Kibana and Graylog), a lightning fast distributed database, that can’t do a join and might crash if a message is a string or is not a string. POSTMORTEM: WHY YOU SHOULD NOT RE-USE CSR WHEN RENEWINGSEE MORE ONTHEHFTGUY.COM
MAKING BEAUTIFULSOUP PARSING 10 TIMES FASTER BeautifulSoup. 1. 2. soup = BeautifulSoup (r.content, 'http.parser') Translation: 4.28 seconds to download 4 pages ( requests.api + requests.sessions) 7.92 seconds to parse 4 pages ( bs4.__init__) The HTML parsing is extremely slow indeed. Looks like it’s spending 7 seconds just to detect the character set of the document. CONFIGURING TIMEOUTS IN HAPROXY THE DIFFERENCE BETWEEN ID_TOKEN AND ACCESS_TOKEN IN OPENID As a developer, you need the user identifier. Typically a primary key, an email or an employee ID. The two tokens are mostly equivalent, except for the mail field. Let’s assume that users are referenced by email in this example. The access_token is worthless here. It’smissing the email.
CRACKING THE HACKERRANK TEST: 100% SCORE MADE EASY Foreword It's well known that most programmers wannabes can't code their way out of a paper bag. As a consequence, the tech industry is pushing for longer, harder and evermore extreme screening. The whiteboard interview has been the standard for a while, followed by puzzles , then FizzBuzz. The latest fad is HackerRank.It's
BORKED WINDOWS 10 UPDATE 2020-08-31: RESET SOUND SETTINGS Borked Windows 10 Update 2020-08-31: Reset Sound Settings. I’ve decided to open a recurrent series “ Borked Windows Update ” to narrate my adventures with broken Windows updates. In the 2 years I’ve been using Windows 10 (had to upgrade because Windows 7 was nearing end of life), I personally faced about 5 major-breakingupdates
TECHNICAL DEBT DOESN’T EXIST After decades of software engineering, I came to the professional conclusion that technical debt doesn't exist. Oh I have seen software rewritten countless times under the pretense of technical debt, doesn't mean there was technical debt unless "technical debt" is defined as "any existing code". Software is unique in the way thateverything that was
MAKING BEAUTIFULSOUP PARSING 10 TIMES FASTER BeautifulSoup. 1. 2. soup = BeautifulSoup (r.content, 'http.parser') Translation: 4.28 seconds to download 4 pages ( requests.api + requests.sessions) 7.92 seconds to parse 4 pages ( bs4.__init__) The HTML parsing is extremely slow indeed. Looks like it’s spending 7 seconds just to detect the character set of the document. OPENDJ – THE HFT GUY A bit of history: OpenDJ was originally developed by Sun, who invented Java and the JVM. The story is that used internally OpenDJ as a benchmark during the development of the G1 garbage collector. This is the latest java garbage collector recommended when using more than 8 WILL AIRBNB GO BANKRUPT? AND WHEN? Read the news: Airbnb Racks Up Hundreds of Millions in Losses Due to Coronavirus AirBnb is losing money at the speed or light and that will only get worse over the coming months. The question on everyone's mind is. Will AirBnb die? and more importantly, JWT IS AWESOME: HERE’S WHY Here's my take on JWT* and why I think it's amazing, after having migrated hundreds of applications in JP Morgan from a legacy authentication solution. *JWT = JSON Web Token 1) Pro: JWT is standardized and supported in most languages For applications to have authentication, this requires developers to tightly integrate authentication when writing the BEFORE AND AFTER DOCKER: HOW TO DEPLOY AN Docker is a packaging and deployment system. It allows you to package an application as a "docker image", then deploy it easily on some servers with a single "docker start " command. Packaging an application Packaging an application without Docker A developer pushes a change The CI sees that new code is available. It rebuilds HOW TO PRESENT A GITHUB PROJECT FOR YOUR RESUME Introduction Companies ask for a GitHub profile. Recruiters ask for a GitHub profile. The question "Do you contribute to open-source?" is now one of the most common questions asked in phone screens. If people want a GitHub, we shall give them a GitHub. This article will explain how to present a GitHub project for use CRACKING THE HACKERRANK TEST: 100% SCORE MADE EASY Foreword It's well known that most programmers wannabes can't code their way out of a paper bag. As a consequence, the tech industry is pushing for longer, harder and evermore extreme screening. The whiteboard interview has been the standard for a while, followed by puzzles , then FizzBuzz. The latest fad is HackerRank.It's
HACKERRANK TESTING: A GLIMPSE AT THE COMPANY SIDE HackerRank is an online coding platform. It provides coding tests and questions for companies to screen candidates. We remember the first time we had to do a test (before joining the company), unsure what were the expectations. Later, we were designing new tests (after joining the company), unsure what to expect from candidates. Wedecided
THE HFT GUY
My Experience In Production with: Flask, Bottle, Tornado and Twisted. 27 October 2020. 19 October 2020 thehftguy. 6 Comments. i. 9 Votes. I’ve been using flask, bottle, tornado in production day to day for years, writing plugins for these and maintaining a variety of new or legacy applications that happened in Big Corp TM. FRENCH JUDGE RULES GPL LICENSE TO BE INAPPLICABLE INSEE MORE ONTHEHFTGUY.COM
A HAVEN FOR FRAUD AND STOLEN CARDS… OR HOW MY WIFE WASSEE MORE ONTHEHFTGUY.COM
WHAT AES CIPHERS TO USE BETWEEN CBC, GCM, CCM, CHACHA-POLYSEE MORE ONTHEHFTGUY.COM
THE DIFFERENCES BETWEEN SPLUNK, KIBANA AND GRAYLOG. Choice is open between: Splunk, a tool with incredible joining and charting capability, that is dog slow and cost a fortune. ElasticSearch (both Kibana and Graylog), a lightning fast distributed database, that can’t do a join and might crash if a message is a string or is not a string. POSTMORTEM: WHY YOU SHOULD NOT RE-USE CSR WHEN RENEWINGSEE MORE ONTHEHFTGUY.COM
MAKING BEAUTIFULSOUP PARSING 10 TIMES FASTER BeautifulSoup. 1. 2. soup = BeautifulSoup (r.content, 'http.parser') Translation: 4.28 seconds to download 4 pages ( requests.api + requests.sessions) 7.92 seconds to parse 4 pages ( bs4.__init__) The HTML parsing is extremely slow indeed. Looks like it’s spending 7 seconds just to detect the character set of the document. CONFIGURING TIMEOUTS IN HAPROXY THE DIFFERENCE BETWEEN ID_TOKEN AND ACCESS_TOKEN IN OPENID As a developer, you need the user identifier. Typically a primary key, an email or an employee ID. The two tokens are mostly equivalent, except for the mail field. Let’s assume that users are referenced by email in this example. The access_token is worthless here. It’smissing the email.
CRACKING THE HACKERRANK TEST: 100% SCORE MADE EASY Foreword It's well known that most programmers wannabes can't code their way out of a paper bag. As a consequence, the tech industry is pushing for longer, harder and evermore extreme screening. The whiteboard interview has been the standard for a while, followed by puzzles , then FizzBuzz. The latest fad is HackerRank.It's
THE HFT GUY
My Experience In Production with: Flask, Bottle, Tornado and Twisted. 27 October 2020. 19 October 2020 thehftguy. 6 Comments. i. 9 Votes. I’ve been using flask, bottle, tornado in production day to day for years, writing plugins for these and maintaining a variety of new or legacy applications that happened in Big Corp TM. FRENCH JUDGE RULES GPL LICENSE TO BE INAPPLICABLE INSEE MORE ONTHEHFTGUY.COM
A HAVEN FOR FRAUD AND STOLEN CARDS… OR HOW MY WIFE WASSEE MORE ONTHEHFTGUY.COM
WHAT AES CIPHERS TO USE BETWEEN CBC, GCM, CCM, CHACHA-POLYSEE MORE ONTHEHFTGUY.COM
THE DIFFERENCES BETWEEN SPLUNK, KIBANA AND GRAYLOG. Choice is open between: Splunk, a tool with incredible joining and charting capability, that is dog slow and cost a fortune. ElasticSearch (both Kibana and Graylog), a lightning fast distributed database, that can’t do a join and might crash if a message is a string or is not a string. POSTMORTEM: WHY YOU SHOULD NOT RE-USE CSR WHEN RENEWINGSEE MORE ONTHEHFTGUY.COM
MAKING BEAUTIFULSOUP PARSING 10 TIMES FASTER BeautifulSoup. 1. 2. soup = BeautifulSoup (r.content, 'http.parser') Translation: 4.28 seconds to download 4 pages ( requests.api + requests.sessions) 7.92 seconds to parse 4 pages ( bs4.__init__) The HTML parsing is extremely slow indeed. Looks like it’s spending 7 seconds just to detect the character set of the document. CONFIGURING TIMEOUTS IN HAPROXY THE DIFFERENCE BETWEEN ID_TOKEN AND ACCESS_TOKEN IN OPENID As a developer, you need the user identifier. Typically a primary key, an email or an employee ID. The two tokens are mostly equivalent, except for the mail field. Let’s assume that users are referenced by email in this example. The access_token is worthless here. It’smissing the email.
CRACKING THE HACKERRANK TEST: 100% SCORE MADE EASY Foreword It's well known that most programmers wannabes can't code their way out of a paper bag. As a consequence, the tech industry is pushing for longer, harder and evermore extreme screening. The whiteboard interview has been the standard for a while, followed by puzzles , then FizzBuzz. The latest fad is HackerRank.It's
BORKED WINDOWS 10 UPDATE 2020-08-31: RESET SOUND SETTINGS Borked Windows 10 Update 2020-08-31: Reset Sound Settings. I’ve decided to open a recurrent series “ Borked Windows Update ” to narrate my adventures with broken Windows updates. In the 2 years I’ve been using Windows 10 (had to upgrade because Windows 7 was nearing end of life), I personally faced about 5 major-breakingupdates
TECHNICAL DEBT DOESN’T EXIST After decades of software engineering, I came to the professional conclusion that technical debt doesn't exist. Oh I have seen software rewritten countless times under the pretense of technical debt, doesn't mean there was technical debt unless "technical debt" is defined as "any existing code". Software is unique in the way thateverything that was
MAKING BEAUTIFULSOUP PARSING 10 TIMES FASTER BeautifulSoup. 1. 2. soup = BeautifulSoup (r.content, 'http.parser') Translation: 4.28 seconds to download 4 pages ( requests.api + requests.sessions) 7.92 seconds to parse 4 pages ( bs4.__init__) The HTML parsing is extremely slow indeed. Looks like it’s spending 7 seconds just to detect the character set of the document. OPENDJ – THE HFT GUY A bit of history: OpenDJ was originally developed by Sun, who invented Java and the JVM. The story is that used internally OpenDJ as a benchmark during the development of the G1 garbage collector. This is the latest java garbage collector recommended when using more than 8 WILL AIRBNB GO BANKRUPT? AND WHEN? Read the news: Airbnb Racks Up Hundreds of Millions in Losses Due to Coronavirus AirBnb is losing money at the speed or light and that will only get worse over the coming months. The question on everyone's mind is. Will AirBnb die? and more importantly, JWT IS AWESOME: HERE’S WHY Here's my take on JWT* and why I think it's amazing, after having migrated hundreds of applications in JP Morgan from a legacy authentication solution. *JWT = JSON Web Token 1) Pro: JWT is standardized and supported in most languages For applications to have authentication, this requires developers to tightly integrate authentication when writing the BEFORE AND AFTER DOCKER: HOW TO DEPLOY AN Docker is a packaging and deployment system. It allows you to package an application as a "docker image", then deploy it easily on some servers with a single "docker start " command. Packaging an application Packaging an application without Docker A developer pushes a change The CI sees that new code is available. It rebuilds HOW TO PRESENT A GITHUB PROJECT FOR YOUR RESUME Introduction Companies ask for a GitHub profile. Recruiters ask for a GitHub profile. The question "Do you contribute to open-source?" is now one of the most common questions asked in phone screens. If people want a GitHub, we shall give them a GitHub. This article will explain how to present a GitHub project for use CRACKING THE HACKERRANK TEST: 100% SCORE MADE EASY Foreword It's well known that most programmers wannabes can't code their way out of a paper bag. As a consequence, the tech industry is pushing for longer, harder and evermore extreme screening. The whiteboard interview has been the standard for a while, followed by puzzles , then FizzBuzz. The latest fad is HackerRank.It's
HACKERRANK TESTING: A GLIMPSE AT THE COMPANY SIDE HackerRank is an online coding platform. It provides coding tests and questions for companies to screen candidates. We remember the first time we had to do a test (before joining the company), unsure what were the expectations. Later, we were designing new tests (after joining the company), unsure what to expect from candidates. Wedecided
THE HFT GUY
My Experience In Production with: Flask, Bottle, Tornado and Twisted. 27 October 2020. 19 October 2020 thehftguy. 6 Comments. i. 9 Votes. I’ve been using flask, bottle, tornado in production day to day for years, writing plugins for these and maintaining a variety of new or legacy applications that happened in Big Corp TM. POSTMORTEM: WHY YOU SHOULD NOT RE-USE CSR WHEN RENEWINGSEE MORE ONTHEHFTGUY.COM
A HAVEN FOR FRAUD AND STOLEN CARDS… OR HOW MY WIFE WASSEE MORE ONTHEHFTGUY.COM
FRENCH JUDGE RULES GPL LICENSE TO BE INAPPLICABLE INSEE MORE ONTHEHFTGUY.COM
WHAT AES CIPHERS TO USE BETWEEN CBC, GCM, CCM, CHACHA-POLYSEE MORE ONTHEHFTGUY.COM
THE DIFFERENCES BETWEEN SPLUNK, KIBANA AND GRAYLOG. Choice is open between: Splunk, a tool with incredible joining and charting capability, that is dog slow and cost a fortune. ElasticSearch (both Kibana and Graylog), a lightning fast distributed database, that can’t do a join and might crash if a message is a string or is not a string. CONFIGURING TIMEOUTS IN HAPROXY HOW TO PRESENT A GITHUB PROJECT FOR YOUR RESUME Introduction Companies ask for a GitHub profile. Recruiters ask for a GitHub profile. The question "Do you contribute to open-source?" is now one of the most common questions asked in phone screens. If people want a GitHub, we shall give them a GitHub. This article will explain how to present a GitHub project for use OPENDJ – THE HFT GUYSEE MORE ON THEHFTGUY.COM THE DIFFERENCE BETWEEN ID_TOKEN AND ACCESS_TOKEN IN OPENID As a developer, you need the user identifier. Typically a primary key, an email or an employee ID. The two tokens are mostly equivalent, except for the mail field. Let’s assume that users are referenced by email in this example. The access_token is worthless here. It’smissing the email.
THE HFT GUY
My Experience In Production with: Flask, Bottle, Tornado and Twisted. 27 October 2020. 19 October 2020 thehftguy. 6 Comments. i. 9 Votes. I’ve been using flask, bottle, tornado in production day to day for years, writing plugins for these and maintaining a variety of new or legacy applications that happened in Big Corp TM. POSTMORTEM: WHY YOU SHOULD NOT RE-USE CSR WHEN RENEWINGSEE MORE ONTHEHFTGUY.COM
A HAVEN FOR FRAUD AND STOLEN CARDS… OR HOW MY WIFE WASSEE MORE ONTHEHFTGUY.COM
FRENCH JUDGE RULES GPL LICENSE TO BE INAPPLICABLE INSEE MORE ONTHEHFTGUY.COM
WHAT AES CIPHERS TO USE BETWEEN CBC, GCM, CCM, CHACHA-POLYSEE MORE ONTHEHFTGUY.COM
THE DIFFERENCES BETWEEN SPLUNK, KIBANA AND GRAYLOG. Choice is open between: Splunk, a tool with incredible joining and charting capability, that is dog slow and cost a fortune. ElasticSearch (both Kibana and Graylog), a lightning fast distributed database, that can’t do a join and might crash if a message is a string or is not a string. CONFIGURING TIMEOUTS IN HAPROXY HOW TO PRESENT A GITHUB PROJECT FOR YOUR RESUME Introduction Companies ask for a GitHub profile. Recruiters ask for a GitHub profile. The question "Do you contribute to open-source?" is now one of the most common questions asked in phone screens. If people want a GitHub, we shall give them a GitHub. This article will explain how to present a GitHub project for use OPENDJ – THE HFT GUYSEE MORE ON THEHFTGUY.COM THE DIFFERENCE BETWEEN ID_TOKEN AND ACCESS_TOKEN IN OPENID As a developer, you need the user identifier. Typically a primary key, an email or an employee ID. The two tokens are mostly equivalent, except for the mail field. Let’s assume that users are referenced by email in this example. The access_token is worthless here. It’smissing the email.
THE HFT GUY
My Experience In Production with: Flask, Bottle, Tornado and Twisted. 27 October 2020. 19 October 2020 thehftguy. 6 Comments. i. 9 Votes. I’ve been using flask, bottle, tornado in production day to day for years, writing plugins for these and maintaining a variety of new or legacy applications that happened in Big Corp TM. OPENDJ – THE HFT GUY A bit of history: OpenDJ was originally developed by Sun, who invented Java and the JVM. The story is that used internally OpenDJ as a benchmark during the development of the G1 garbage collector. This is the latest java garbage collector recommended when using more than 8 TECHNICAL DEBT DOESN’T EXIST After decades of software engineering, I came to the professional conclusion that technical debt doesn't exist. Oh I have seen software rewritten countless times under the pretense of technical debt, doesn't mean there was technical debt unless "technical debt" is defined as "any existing code". Software is unique in the way thateverything that was
JWT IS AWESOME: HERE’S WHY Here's my take on JWT* and why I think it's amazing, after having migrated hundreds of applications in JP Morgan from a legacy authentication solution. *JWT = JSON Web Token 1) Pro: JWT is standardized and supported in most languages For applications to have authentication, this requires developers to tightly integrate authentication when writing the BORKED WINDOWS 10 UPDATE 2020-08-31: RESET SOUND SETTINGS Borked Windows 10 Update 2020-08-31: Reset Sound Settings. I’ve decided to open a recurrent series “ Borked Windows Update ” to narrate my adventures with broken Windows updates. In the 2 years I’ve been using Windows 10 (had to upgrade because Windows 7 was nearing end of life), I personally faced about 5 major-breakingupdates
MAKING BEAUTIFULSOUP PARSING 10 TIMES FASTER BeautifulSoup. 1. 2. soup = BeautifulSoup (r.content, 'http.parser') Translation: 4.28 seconds to download 4 pages ( requests.api + requests.sessions) 7.92 seconds to parse 4 pages ( bs4.__init__) The HTML parsing is extremely slow indeed. Looks like it’s spending 7 seconds just to detect the character set of the document. PERL IS DYING QUICK. COULD BE EXTINCT BY 2023. Note that both regression curves happen to overlap for COBOL. A stable and predictable language indeed. If we are to believe the linear projections, Perl could be extinct by 2023, a few months after Delphi. COBOL would outlive both by far and go on to 2030. Of course, languages don’t go “extinct”. BEFORE AND AFTER DOCKER: HOW TO DEPLOY AN Docker is a packaging and deployment system. It allows you to package an application as a "docker image", then deploy it easily on some servers with a single "docker start " command. Packaging an application Packaging an application without Docker A developer pushes a change The CI sees that new code is available. It rebuilds CRACKING THE HACKERRANK TEST: 100% SCORE MADE EASY Foreword It's well known that most programmers wannabes can't code their way out of a paper bag. As a consequence, the tech industry is pushing for longer, harder and evermore extreme screening. The whiteboard interview has been the standard for a while, followed by puzzles , then FizzBuzz. The latest fad is HackerRank.It's
IAM BENCHMARK: OPENAM VS WSO2 VS SIMPLESAMLPHP VS Benchmark Setup Test Scenario Users connect again and again forever. The full user profile (name, address, phone, etc) is requested on each connection. This is the preferred mode of testing because it is the most intensive and it is similar to peak load during peak hours. Bottlenecks can be identified. Unreliable components will breakTHE HFT GUY
My Experience In Production with: Flask, Bottle, Tornado and Twisted. 27 October 2020. 19 October 2020 thehftguy. 6 Comments. i. 9 Votes. I’ve been using flask, bottle, tornado in production day to day for years, writing plugins for these and maintaining a variety of new or legacy applications that happened in Big Corp TM. FRENCH JUDGE RULES GPL LICENSE TO BE INAPPLICABLE INSEE MORE ONTHEHFTGUY.COM
A HAVEN FOR FRAUD AND STOLEN CARDS… OR HOW MY WIFE WASSEE MORE ONTHEHFTGUY.COM
TECHNICAL DEBT DOESN’T EXIST After decades of software engineering, I came to the professional conclusion that technical debt doesn't exist. Oh I have seen software rewritten countless times under the pretense of technical debt, doesn't mean there was technical debt unless "technical debt" is defined as "any existing code". Software is unique in the way thateverything that was
POSTMORTEM: WHY YOU SHOULD NOT RE-USE CSR WHEN RENEWINGSEE MORE ONTHEHFTGUY.COM
JWT IS AWESOME: HERE’S WHY Here's my take on JWT* and why I think it's amazing, after having migrated hundreds of applications in JP Morgan from a legacy authentication solution. *JWT = JSON Web Token 1) Pro: JWT is standardized and supported in most languages For applications to have authentication, this requires developers to tightly integrate authentication when writing the WHAT AES CIPHERS TO USE BETWEEN CBC, GCM, CCM, CHACHA-POLYSEE MORE ONTHEHFTGUY.COM
OPENDJ – THE HFT GUYSEE MORE ON THEHFTGUY.COM THE DIFFERENCE BETWEEN ID_TOKEN AND ACCESS_TOKEN IN OPENID As a developer, you need the user identifier. Typically a primary key, an email or an employee ID. The two tokens are mostly equivalent, except for the mail field. Let’s assume that users are referenced by email in this example. The access_token is worthless here. It’smissing the email.
THE INEVITABLE DEMISE OF IBM SOFTLAYER BY AWS Introduction There are many types of cloud providers, public vs private, cheap vs expensive. You can read my article on which provider to pick and when. Today, we're going to talk about the expensive clouds for enterprises, namely SoftLayer, AWS and Google. For reference, IBM SoftLayer is used to order dedicated beefy servers.Including but
THE HFT GUY
My Experience In Production with: Flask, Bottle, Tornado and Twisted. 27 October 2020. 19 October 2020 thehftguy. 6 Comments. i. 9 Votes. I’ve been using flask, bottle, tornado in production day to day for years, writing plugins for these and maintaining a variety of new or legacy applications that happened in Big Corp TM. FRENCH JUDGE RULES GPL LICENSE TO BE INAPPLICABLE INSEE MORE ONTHEHFTGUY.COM
A HAVEN FOR FRAUD AND STOLEN CARDS… OR HOW MY WIFE WASSEE MORE ONTHEHFTGUY.COM
TECHNICAL DEBT DOESN’T EXIST After decades of software engineering, I came to the professional conclusion that technical debt doesn't exist. Oh I have seen software rewritten countless times under the pretense of technical debt, doesn't mean there was technical debt unless "technical debt" is defined as "any existing code". Software is unique in the way thateverything that was
JWT IS AWESOME: HERE’S WHY Here's my take on JWT* and why I think it's amazing, after having migrated hundreds of applications in JP Morgan from a legacy authentication solution. *JWT = JSON Web Token 1) Pro: JWT is standardized and supported in most languages For applications to have authentication, this requires developers to tightly integrate authentication when writing the POSTMORTEM: WHY YOU SHOULD NOT RE-USE CSR WHEN RENEWINGSEE MORE ONTHEHFTGUY.COM
WHAT AES CIPHERS TO USE BETWEEN CBC, GCM, CCM, CHACHA-POLYSEE MORE ONTHEHFTGUY.COM
OPENDJ – THE HFT GUYSEE MORE ON THEHFTGUY.COM THE DIFFERENCE BETWEEN ID_TOKEN AND ACCESS_TOKEN IN OPENID As a developer, you need the user identifier. Typically a primary key, an email or an employee ID. The two tokens are mostly equivalent, except for the mail field. Let’s assume that users are referenced by email in this example. The access_token is worthless here. It’smissing the email.
THE INEVITABLE DEMISE OF IBM SOFTLAYER BY AWS Introduction There are many types of cloud providers, public vs private, cheap vs expensive. You can read my article on which provider to pick and when. Today, we're going to talk about the expensive clouds for enterprises, namely SoftLayer, AWS and Google. For reference, IBM SoftLayer is used to order dedicated beefy servers.Including but
THE HFT GUY
My Experience In Production with: Flask, Bottle, Tornado and Twisted. 27 October 2020. 19 October 2020 thehftguy. 6 Comments. i. 9 Votes. I’ve been using flask, bottle, tornado in production day to day for years, writing plugins for these and maintaining a variety of new or legacy applications that happened in Big Corp TM. ABOUT ME – THE HFT GUY Who Am I? A multi-skilled developer in London, with a solid background in software engineering and a master’s degree. I have a wide range of experiences from software companies, fintech, aerospace, banks, consulting and government projects. I live to build things and I build things for a living. Why "The HFT Guy"? High Frequency Trading BORKED WINDOWS 10 UPDATE 2020-08-31: RESET SOUND SETTINGS Borked Windows 10 Update 2020-08-31: Reset Sound Settings. I’ve decided to open a recurrent series “ Borked Windows Update ” to narrate my adventures with broken Windows updates. In the 2 years I’ve been using Windows 10 (had to upgrade because Windows 7 was nearing end of life), I personally faced about 5 major-breakingupdates
WHAT TAKES LONGER TO BUILD, WINDOWS OR THE GREAT PYRAMID Skip to third sections for charts. On the one hand, the Great Pyramid, one of the 7 Wonders of the world, built thousands of years ago. On the other hand, Microsoft Windows, the operating system running computers around the world. Let's measure how much effort they took to build. The "effort" as a project management WILL AIRBNB GO BANKRUPT? AND WHEN? AirBnb needing few billions cash is a godsend for an investor on a 5-10 years horizon. It is ripe for an hostile takeover from a Vulture Venture Capitalist. Get as much control as possible, lay off one third of the company, freeze headcount for 2-3 years. It will be printingmoney soon enough.
MAKING BEAUTIFULSOUP PARSING 10 TIMES FASTER BeautifulSoup. 1. 2. soup = BeautifulSoup (r.content, 'http.parser') Translation: 4.28 seconds to download 4 pages ( requests.api + requests.sessions) 7.92 seconds to parse 4 pages ( bs4.__init__) The HTML parsing is extremely slow indeed. Looks like it’s spending 7 seconds just to detect the character set of the document. HOW TO PRESENT A GITHUB PROJECT FOR YOUR RESUME Introduction Companies ask for a GitHub profile. Recruiters ask for a GitHub profile. The question "Do you contribute to open-source?" is now one of the most common questions asked in phone screens. If people want a GitHub, we shall give them a GitHub. This article will explain how to present a GitHub project for use CONFIGURING TIMEOUTS IN HAPROXY established. Set additional check timeout, but only after a connection has been already. If set, haproxy uses min (“timeout connect”, “inter”) as a connect timeout. for check and “timeout check” as an additional read timeout. The “min” is. used so that peoplerunning with
250 GB/DAY OF LOGS WITH GRAYLOG: LESSONS LEARNED Storage (as measured in production) 138 906 326 logs per day (averaged over the last 7 days) 2200 GB used, for 9 days of data. 1800 bytes/log in average. Our current logs require 250 GB of space per day. 12 TB will allow for 36 days of log history (at CRACKING THE HACKERRANK TEST: 100% SCORE MADE EASY Foreword It's well known that most programmers wannabes can't code their way out of a paper bag. As a consequence, the tech industry is pushing for longer, harder and evermore extreme screening. The whiteboard interview has been the standard for a while, followed by puzzles , then FizzBuzz. The latest fad is HackerRank.It's
THE HFT GUY
My Experience In Production with: Flask, Bottle, Tornado and Twisted. 27 October 2020. 19 October 2020 thehftguy. 6 Comments. i. 9 Votes. I’ve been using flask, bottle, tornado in production day to day for years, writing plugins for these and maintaining a variety of new or legacy applications that happened in Big Corp TM. FRENCH JUDGE RULES GPL LICENSE TO BE INAPPLICABLE INSEE MORE ONTHEHFTGUY.COM
A HAVEN FOR FRAUD AND STOLEN CARDS… OR HOW MY WIFE WASSEE MORE ONTHEHFTGUY.COM
TECHNICAL DEBT DOESN’T EXIST After decades of software engineering, I came to the professional conclusion that technical debt doesn't exist. Oh I have seen software rewritten countless times under the pretense of technical debt, doesn't mean there was technical debt unless "technical debt" is defined as "any existing code". Software is unique in the way thateverything that was
POSTMORTEM: WHY YOU SHOULD NOT RE-USE CSR WHEN RENEWINGSEE MORE ONTHEHFTGUY.COM
JWT IS AWESOME: HERE’S WHY Here's my take on JWT* and why I think it's amazing, after having migrated hundreds of applications in JP Morgan from a legacy authentication solution. *JWT = JSON Web Token 1) Pro: JWT is standardized and supported in most languages For applications to have authentication, this requires developers to tightly integrate authentication when writing the WHAT AES CIPHERS TO USE BETWEEN CBC, GCM, CCM, CHACHA-POLYSEE MORE ONTHEHFTGUY.COM
OPENDJ – THE HFT GUYSEE MORE ON THEHFTGUY.COM THE DIFFERENCE BETWEEN ID_TOKEN AND ACCESS_TOKEN IN OPENID As a developer, you need the user identifier. Typically a primary key, an email or an employee ID. The two tokens are mostly equivalent, except for the mail field. Let’s assume that users are referenced by email in this example. The access_token is worthless here. It’smissing the email.
THE INEVITABLE DEMISE OF IBM SOFTLAYER BY AWS Introduction There are many types of cloud providers, public vs private, cheap vs expensive. You can read my article on which provider to pick and when. Today, we're going to talk about the expensive clouds for enterprises, namely SoftLayer, AWS and Google. For reference, IBM SoftLayer is used to order dedicated beefy servers.Including but
THE HFT GUY
My Experience In Production with: Flask, Bottle, Tornado and Twisted. 27 October 2020. 19 October 2020 thehftguy. 6 Comments. i. 9 Votes. I’ve been using flask, bottle, tornado in production day to day for years, writing plugins for these and maintaining a variety of new or legacy applications that happened in Big Corp TM. FRENCH JUDGE RULES GPL LICENSE TO BE INAPPLICABLE INSEE MORE ONTHEHFTGUY.COM
A HAVEN FOR FRAUD AND STOLEN CARDS… OR HOW MY WIFE WASSEE MORE ONTHEHFTGUY.COM
TECHNICAL DEBT DOESN’T EXIST After decades of software engineering, I came to the professional conclusion that technical debt doesn't exist. Oh I have seen software rewritten countless times under the pretense of technical debt, doesn't mean there was technical debt unless "technical debt" is defined as "any existing code". Software is unique in the way thateverything that was
POSTMORTEM: WHY YOU SHOULD NOT RE-USE CSR WHEN RENEWINGSEE MORE ONTHEHFTGUY.COM
JWT IS AWESOME: HERE’S WHY Here's my take on JWT* and why I think it's amazing, after having migrated hundreds of applications in JP Morgan from a legacy authentication solution. *JWT = JSON Web Token 1) Pro: JWT is standardized and supported in most languages For applications to have authentication, this requires developers to tightly integrate authentication when writing the WHAT AES CIPHERS TO USE BETWEEN CBC, GCM, CCM, CHACHA-POLYSEE MORE ONTHEHFTGUY.COM
OPENDJ – THE HFT GUYSEE MORE ON THEHFTGUY.COM THE DIFFERENCE BETWEEN ID_TOKEN AND ACCESS_TOKEN IN OPENID As a developer, you need the user identifier. Typically a primary key, an email or an employee ID. The two tokens are mostly equivalent, except for the mail field. Let’s assume that users are referenced by email in this example. The access_token is worthless here. It’smissing the email.
THE INEVITABLE DEMISE OF IBM SOFTLAYER BY AWS Introduction There are many types of cloud providers, public vs private, cheap vs expensive. You can read my article on which provider to pick and when. Today, we're going to talk about the expensive clouds for enterprises, namely SoftLayer, AWS and Google. For reference, IBM SoftLayer is used to order dedicated beefy servers.Including but
THE HFT GUY
My Experience In Production with: Flask, Bottle, Tornado and Twisted. 27 October 2020. 19 October 2020 thehftguy. 6 Comments. i. 9 Votes. I’ve been using flask, bottle, tornado in production day to day for years, writing plugins for these and maintaining a variety of new or legacy applications that happened in Big Corp TM. ABOUT ME – THE HFT GUY Who Am I? A multi-skilled developer in London, with a solid background in software engineering and a master’s degree. I have a wide range of experiences from software companies, fintech, aerospace, banks, consulting and government projects. I live to build things and I build things for a living. Why "The HFT Guy"? High Frequency Trading BORKED WINDOWS 10 UPDATE 2020-08-31: RESET SOUND SETTINGS Borked Windows 10 Update 2020-08-31: Reset Sound Settings. I’ve decided to open a recurrent series “ Borked Windows Update ” to narrate my adventures with broken Windows updates. In the 2 years I’ve been using Windows 10 (had to upgrade because Windows 7 was nearing end of life), I personally faced about 5 major-breakingupdates
WHAT TAKES LONGER TO BUILD, WINDOWS OR THE GREAT PYRAMID Skip to third sections for charts. On the one hand, the Great Pyramid, one of the 7 Wonders of the world, built thousands of years ago. On the other hand, Microsoft Windows, the operating system running computers around the world. Let's measure how much effort they took to build. The "effort" as a project management WILL AIRBNB GO BANKRUPT? AND WHEN? AirBnb needing few billions cash is a godsend for an investor on a 5-10 years horizon. It is ripe for an hostile takeover from a Vulture Venture Capitalist. Get as much control as possible, lay off one third of the company, freeze headcount for 2-3 years. It will be printingmoney soon enough.
MAKING BEAUTIFULSOUP PARSING 10 TIMES FASTER BeautifulSoup. 1. 2. soup = BeautifulSoup (r.content, 'http.parser') Translation: 4.28 seconds to download 4 pages ( requests.api + requests.sessions) 7.92 seconds to parse 4 pages ( bs4.__init__) The HTML parsing is extremely slow indeed. Looks like it’s spending 7 seconds just to detect the character set of the document. HOW TO PRESENT A GITHUB PROJECT FOR YOUR RESUME Introduction Companies ask for a GitHub profile. Recruiters ask for a GitHub profile. The question "Do you contribute to open-source?" is now one of the most common questions asked in phone screens. If people want a GitHub, we shall give them a GitHub. This article will explain how to present a GitHub project for use CONFIGURING TIMEOUTS IN HAPROXY established. Set additional check timeout, but only after a connection has been already. If set, haproxy uses min (“timeout connect”, “inter”) as a connect timeout. for check and “timeout check” as an additional read timeout. The “min” is. used so that peoplerunning with
250 GB/DAY OF LOGS WITH GRAYLOG: LESSONS LEARNED Storage (as measured in production) 138 906 326 logs per day (averaged over the last 7 days) 2200 GB used, for 9 days of data. 1800 bytes/log in average. Our current logs require 250 GB of space per day. 12 TB will allow for 36 days of log history (at CRACKING THE HACKERRANK TEST: 100% SCORE MADE EASY Foreword It's well known that most programmers wannabes can't code their way out of a paper bag. As a consequence, the tech industry is pushing for longer, harder and evermore extreme screening. The whiteboard interview has been the standard for a while, followed by puzzles , then FizzBuzz. The latest fad is HackerRank.It's
Skip to content
__Open Menu
* __Home
* About Me
* Contact
* Privacy Policy
THE HFT GUY
A DEVELOPER IN LONDON MAJOR BUG IN GLIBC IS KILLING APPLICATIONS WITH A MEMORY LIMIT __21 May 202020 May 2020 __thehftguy __Leave a commenti
4 Votes
This is the story of a debugging case with a happy ending. TL;DR THIS IS A BUG IN THE GLIBC MALLOC(). IT MAINLY AFFECTS 64 BITS MULTI-THREADING APPLICATIONS. WITH A SPECIAL MENTION TO JAVA APPLICATIONS BECAUSE THE JVM SEEMS TO ALWAYS TRIGGER THE WORST CASE. THE WRITE UP IS QUITE LONG, SKIP TO THIRD SECTION GLIBC MALLOC() IF YOU JUST WANT THE ENDING.INTRODUCTION
A developer reported an issue in one of their service and asked for some help to investigate. Sure. Digging into the service logs, I noticed that there were no logs for a handful of instances. It’s barely noticeable, only a few percents of the normal activity goes there and is not accounted for. I TRACKED THIS DOWN TO PAIR OF DEV AND PRODUCTION HOSTS, PART OF LARGER CLUSTERS WITH TENS OF HOSTS. How come services that should be running there are not? More worrying there were no failure notifications, or alerts, or issues reported byusers?
This is a bank and those are fairly critical. These cannot fail nor bemissing.
To the extent of my knowledge they have only failed twice in recent years. The first time due to a bug happening in rare circumstances in one region, the second time due to the complete and sudden loss of adatacenter.
In both cases it took a significant amount of time to understand the root cause (almost one hour to conclude with certainty that a datacenter has exploded? What?!). Both were mostly non-events as the failover kicked in within 30 seconds, re-routing most activity to the other side of the world, long before anybody could realize anything happened or do anything about it. Disaster recovery at it’s peak. What are the odds that we are looking at the soon-to-be third issue, except this time we’re catching it early before other have noticed? Gathering another coworker and back into serious debugging. AFTER CAREFUL CHECKING. THE ABSENT JOBS ARE ALL RUNNING AND WORKING WELL, WHICH IS REASSURING. HOWEVER THEY ARE “GHOSTS” ON TODAY, FOR LACK OF A BETTER WORD. There is no trace of activity or debug logs. HOW IS THAT POSSIBLE? LET’S INVESTIGATE THE HOSTS AND FIGURE OUT WHAT’S WRONG WITH THEM IN PARTICULAR. It doesn’t take long to realize that the hosts are missing pieces around logging/monitoring. One of the easier bits to investigate further might be one of the Logstash agents. It is not running on any of the affected machines. The configuration looks good. Run a configuration check and it passes. There are starting logs from yesterday so it must have worked at somepoint.
LET’S TRY TO RESTART IT. DONE. Odd. It is running fine now. Back filling data that was missing. Why would it be dead in the first place? DATA DISAPPEAR AFTER A FEW MINUTES. WHAT? DOUBLE CHECKING. THERE IS NOMORE PROCESS.
Restart it again, while paying more attention. It starts and dies one minute later. Repeat one more time, same. VICTORY, WE KNOW FOR SURE SOMETHING IS BROKEN AND WE HAVE A REPRO! HAVING A REPRO IS THE MOST CRITICAL INGREDIENT TO THE RESOLUTION OF ATRICKY BUG.
WHAT’S CHANGED?
There are no meaningful error indicating a configuration issue or an application issue or a JVM issue. Let’s restart it again keeping a close look at the memory consumption. It’s going up and up to multiple gigabytes and die. WORKING SOFTWARE DOESN’T STOP WORKING BY ITSELF, BUT DUE TO SOME CHANGES IN ITS CONFIGURATION OR ENVIRONMENT. WHAT CHANGED RECENTLY? For context, the firm is mono repo with thousands of developers and hundreds of thousands of jobs deployed every day. Almost exclusively apython shop.
THERE IS ONE CHANGE HAPPENING FIRMWIDE OVER THE RECENT WEEKS THAT MAY BE RELATED. THE FIRM IS DEPRECATING 32 BITS. LITERALLY EVERYTHING THATIS 32 BITS.
IF 2020 IS THE YEAR PYTHON 2 MUST DIE, THEN 2021 IS THE YEAR 32 BITSMUST DIE.
This makes perfect sense. Server operating systems have been 64 bits for two decades, desktops for one. Many applications, databases and middleware can only run 64 bits because they require more than 4 GB of memory. Major libraries are dropping 32 bits support. ONE CHALLENGE WITH TRANSITIONING FROM 32 BITS TO 64 BITS IS APPLICATIONS WITH MEMORY LEAKS COMBINED WITH THE LACK OF MEMORYLIMITS.
In a 32 bits world, a leaky application can consume more and more memory until it’s approaching the 4 GB memory limit adressable by 32bits and crash.
In a 64 bits world, a leaky application can consume more and more memory with no limit, exhausting resources on the host and crashing other tasks. This is a major recurring source of issues with deployingapplications.
THE SOLUTION IS ALMOST TRIVIAL, SET A MEMORY LIMIT BY DEFAULT. LET’S SAY 8 GB. It’s more than what was available in 32 bits (4GB) with a large margin for extra overheard. SURE ENOUGH. THE PLANNED DEPRECIATION OF 32 BITS CAME ALONG -DEFAULT- MEMORY LIMITS ON EVERY PROCESS. Well announced and staged gradually. Of course there were a couple services that blew up after that and they were immediately fixed by setting a memory limit of 50 GB or whatever high number. Urgency did not call for accuracy. Back to our few dying services. What do they have in common? THE HOSTS FIT THE CRITERIA FOR EARLY STAGING. THEY MUST HAVE RECEIVED THEUPDATE.
QUICK CHECK. YES, THE DEPLOYED PROCESSES GET A MEMORY LIMIT. Again, restart a failing process and monitor carefully, it works fine for about one minute, with the memory increasing, then dies. WHAT ARE THE ODDS THAT IT IS KILLED WHEN REACHING THE MEMORY LIMIT? ONE PROBLEM THOUGH. THERE IS NO WAY THAT JOB CAN APPROACH ANYWHERE NEAR 4 GB OF MEMORY. THIS IS A JAVA APPLICATION AND THE JVM PREALLOCATES MEMORY ON STARTUP, IT IS CONFIGURED TO USE EXACTLY 1024 MB OF HEAP -XMS1024M -XMX1024M.GLIBC MALLOC()
WHERE DOES THE EXTRA MEMORY COMES FROM? Long story short, this is due to a bug in malloc(). Well, it’s not a bug it’s a feature. MALLOC() PREALLOCATES LARGE CHUNKS OF MEMORY, PER THREAD. THIS IS MEANT AS A PERFORMANCE OPTIMIZATION, TO REDUCE MEMORY CONTENTION IN HIGHLY THREADED APPLICATIONS. IN 32 BITS RUNTIME, IT CAN PREALLOCATE UP TO 2 * 64 MB * CORES. IN 64 BITS RUNTIME, IT CAN PREALLOCATE UP TO 8 * 64 MB * CORES. On a typical physical server, dual Xeon CPU with a terabyte of RAM. The core count is easily 40 or above. 10 cores * 2 CPU * 2 for hyperthreading.
THIS MEANS A PREALLOCATION OF UP TO 20 GB OF RAM IN THE PROCESS. How much exactly depends on the application, but as a rule of thumb it requires a multi-threaded application to have the most. For some reasons the JVM always seem to trigger the worst case. You asked for a belt so we prepared you one thousand just in case. THE ALLOCATED MEMORY COUNTS TOWARD THE MEMORY LIMIT. That’s why the process fails by dying after one minute, it’s allocating/deallocating memory in different threads, triggering more and more preallocation as a side effect, until it reaches the limit and gets terminated. IT’S A NASTY BUG BETWEEN THE PREALLOCATION IN GLIBC MALLOC() AND THE MEMORY LIMIT IN RLIMIT. ANY APPLICATION THAT HAS A LOW RLIMIT (TENS OF GIGABYTES OR LESS) IS AT RISK TO HIT THIS BUG AND BE TERMINATED.HOW TO MITIGATE?
PREALLOCATION BEHAVIOR CAN BE ADJUSTED WITH THE ENVIRONMENT VARIABLEMALLOC_ARENA_MAX=4
A deployment system that sets a memory limit must set this variable too. The major schedulers and cloud hosting have mitigations out-of-the-box, see Herokuor Hadoop YARN .
A value of 1 can disable it, but is likely to reduce performance by a few percents, so not recommended. Interestingly enough, this is supposed to be a performance optimization but the few benchmarks on the internet all show a net degradation of performance as the value goes up in the 2 digits (from around 10 to higher). The default in the hundreds seems out of whack (512 arenas with 40 cores). EDIT: found after the write up. The issue is referenced in a glibc wiki page from 2013 Todo/Enhancing_malloc “_Throttle number of arenas based on process rlimit on process startup and/or everytime RLIMIT_AS is modified._“SHARE THIS ARTICLE:
* Click to share on Twitter (Opens in new window) * Click to share on Reddit (Opens in new window) * Click to share on Facebook (Opens in new window) * Click to share on LinkedIn (Opens in new window) * Click to share on Hacker News (Opens in new window) * Click to email this to a friend (Opens in new window)* More
*
* Click to share on Pinterest (Opens in new window) * Click to share on Pocket (Opens in new window)*
* Click to share on Telegram (Opens in new window) * Click to share on WhatsApp (Opens in new window)*
* Click to share on Skype (Opens in new window) * Click to share on Tumblr (Opens in new window)*
* Click to print (Opens in new window)*
LIKE THIS:
Like Loading...
POSTS NAVIGATION
__Older Posts
Older posts
MOST POPULAR
*
Cracking the HackerRank Test: 100% score made easy*
How to present a GitHub project for your resume*
250 GB/day of logs with Graylog: Lessons Learned*
How to export Amazon EC2 instances to a CSV file*
Docker in Production: A History of Failure*
HAProxy vs nginx: Why you should NEVER use nginx for load balancing!*
The Difference Between id_token and access_token in OpenID ConnectRECOMMENDED BOOKS
Docker in Practice
:
This book is a gem to help you understand Docker and use iteffectively.
NEWSLETTER
Follow this blog. Get news about the cloud and the latest devopstools.
Follow
NEWEST ARTICLES
* Major Bug in glibc is Killing Applications With a Memory Limit * What AES ciphers to use between CBC, GCM, CCM, Chacha-Poly? * Will AirBnb Go Bankrupt? and When? * Postmortem: Why you should not re-use CSR when renewingSSL certificates
* JWT is Awesome: Here’s WhySITE STATISTICS
* 1,565,802 views
_Back to top_
Post to
Cancel
* Follow
*
* The HFT Guy
* Customize
* Follow
* Sign up
* Log in
* Report this content * Manage subscriptions* Collapse this bar
%d bloggers like this: Send to Email Address Your Name Your Email AddressCancel
Post was not sent - check your email addresses! Email check failed, please try again Sorry, your blog cannot share posts by email.Details
Copyright © 2024 ArchiveBay.com. All rights reserved. Terms of Use | Privacy Policy | DMCA | 2021 | Feedback | Advertising | RSS 2.0