My Life

Took a look and discovered my last life update on here was two years ago today. Here is an overdue update.
Quite a bit has happened.

Activity:

Sleep - I sleep even more than I did before now averaging 7 hours 46 minutes per 24 hour period. My lucid dreams have decreased as I have not been putting in the effort to have them. I have been procrastinating on getting back to doing that, but I think I'm at a turning point in my life right now to move back towards good habits.

Physical exercise - I was doing well with this, exercising 2-3 times a week for about 3 years. I was just on the verge of having a 6-pack again. Unfortunately, for the last 7 months I've completely stopped and it's certainly one of the most important aspects of my life that needs to be revived.

Brain exercise - I've been learning piano using a number of online tools and have been considering getting an actual instructor for weekly lessons. I still do Anki, although not as frequently. So my efficiency on memorization of new items and even recall is less than optimal. I learned how to type one-handed with my left hand at about 40 wpm. I though the efficiency of not having to leave the mouse while I use my keyboard for certain applications such as spreadsheets would be worth it. I have discovered my actual use cases where it's more efficient to not type 100+ wpm using both hands are very few.

Drugs - I'm regularly addicted to caffeine only taking a break a couple times a year for 2-6 weeks. Nicotine I just couldn't get into on a regular basis. I find the benefits so short lived that I have to do it too frequently to be of value. Modafinil I still take on occasion for days I know will be particularly rough. I'd say my use of hallucinogens was at an all time high in 2015 and I've really come down from that this year. I don't really have an opinion on whether that's a positive/negative change.

Diet - I eat 4-5 meals a day. My diet has always been a weak point and I've successfully replaced almost all fast food meals with soylent to compensate. I've expanded my very limited and picky food choices to a larger set by forcing myself to try a large number of new foods or retry foods I had previously decided I didn't like. Most foods I didn't like, but some I did and my tolerance for different food in general is significantly wider.

Travel - This is a new category for me. I was and am very much a workaholic. Vacation and travel wasn't in my books very much, so a life change for me was realizing how important this truly is. Just this year I've traveled out of the country several times and even some new cities within the US. I've made a greater effort on exploration and leaving my comfort zone. Just a small amount of travel really does stretch your perception and understanding of just how tremendous a scope the world provides. This is something I plan on very much doing more of with a trip being planned for Japan and Korea in January.
I'll put Burning Man in this category as well. This year I attended my first one. I would say it was worth it and I am willing to go again, but I'm not sure if I would go out of the way to attend again.

Work - Since my last update I've had two companies that I have founded get acquired. One was for seven figures and the other for eight figures. In complete honesty, neither of them turned out anywhere close to the ideal or even slightly optimistic end result, but with the amount of rigorous work put in and the highly concentrated number of lessons I learned over the past 5 years I'm still proud. I feel very confident that my next ventures will be far more successful with the knowledge and skills that I've accumulated. I have ordered a Tesla Model S with all the nifty upgrades as a reward for myself.
I have one other company that I co-founded still in the works and rapidly growing which I have high hopes for as well.

__________________________________________________

Future
- Get back into a pattern of discipline and accountability where I exercise regularly, lucid dream more frequently, and do brain exercises every day.
- Become highly proficient at piano.
- The last two years I've played less and less poker during the WSOP. I enjoy playing still, but I no longer feel the urge to do so. I plan on playing the Main Event next summer, but if it seems at all like a chore I will likely give up poker.
- Figure out what my next company will be and then grow it using everything I've gained so far.
- Get to a billion

Feedback
- If anything since my last update, I've learned that for how intelligent and adaptable I think I am, I really know very little. I'll gladly hear out feedback. Especially from a community like this that is so centered on optimization.

0 votes

Comments (17)

Some background:
I played poker pretty much full time from 2007-2009. I played about 3 million hands, mostly cash games, over 10,000 tournaments, and had done a great deal of analysis on my play. At the end of 2009, even though I was doing better than ever in poker, I felt like the game had become a job.

After doing research and speaking with a bunch of botters I knew it was possible to be fairly profitable. Already, many of the highest performing low-stakes SNG players on Sharkscope were bots. My long term goal was to build a self-improving botting system that could take on any poker game of any type.

Having spent such an immense amount of time analyzing the game and my play, I felt like utilizing all the information to write an AI would be the best use of that knowledge. I spent a good chunk of 2010 both writing an AI and setting up a botting system that would use that AI to play real money games online. I basically had my bots play until Poker Black Friday. In that time hundreds of thousands of SNGs and over 10 million hands were played.


So how does a bot work?
You must completely automate everything a human would do. Open the client, log in, pick the games using the filters and clicking through the results. Approve or close popups. Join games. Play games. Leave games. Close client.


Botting undetected:
Unfortunately most poker sites terms of service disallowed the use of bots with confiscation of funds and account closure as penalties for violation. The biggest and best sites to play on had these terms, so the first order of business is how to play and not get caught.

There were a number of mechanisms sites employed to enforce these terms including:
Resizing tables
Captchas
Different images for the same cards (although inperceptible to humans)
Pop-up boxes with a human asking a simple question
Scanning running software
Tracking mouse clicks
Checks for non-human play schedules

Also, when pokertableratings started getting popular and allowing the purchasing of hand histories, players themselves were finding and reporting bots by auditing play stats.


After looking at what tools were currently available I ended up choosing a piece of open source software called OpenHoldem and forking it for my own purposes. OpenHoldem had a tool called a TableMap that allowed you to capture images and text from a poker table and translate them into variables that could be used for the logic. Since this software couldn't be made visible to the poker clients being used, I ran poker clients in a headless virtual machine and output the display to a separate computer. Within the virtual machine I used software to place tables at specific resolutions and locations on the screen. A friend helped me write a program that turned the graphic windows from the virtual machine's output into their own separate windows that OpenHoldem could lock to and use the appropriate table map for. Since only the graphic display is being output from the virtual machine, there isn't even a remote desktop client such as VNC or Teamviewer running within.

So far this solves poker clients detecting running software and the resizing of tables. For images being slightly different, I just had to make very robust table map sets where they could account for a certain degree of error and utilize fuzzy fonts. For pop-ups and captchas I had alerts pop-up when there was a certain degree of deviation and I could manually take over for a minute to correct the issue. For the first few months I was always within range of a computer that would give an alert so I could respond to an anomaly. After an account had successfully solved a captcha the risk was lowered and I could feel safe going out to dinner or taking a nap without having to actively monitor play.

The same program that generated the graphic display windows for tables from the virtual machine would accept incoming clicks from OpenHoldem for when an action needed to be done and then would convey those clicks into the virtual machine. Since there were regions where buttons existed, using certain distribution rules, the location on the button could be randomly clicked so as to emulate human clicking behavior.

As far as bots not being caught for their measurably similar behavior, I gave them different personalities. Logic was slightly tweaked for more/less aggressive play in certain areas or overall. For example I could just slightly increase the aggression factor for a player and across every possible scenario they would take the slightly more aggressive approach. I could pick just a few stats such as call preflop raise and 3bet post-flop and tweak them by a percentage. In many cases, this made the bot play less optimally, but different enough to be unique. If a bot unexpectedly performed better in certain areas, I could figure out why and use that for future builds.


Running multiple accounts:
The most obvious advantage of a bot playing is that it can play endlessly at its most optimal settings. You couldn't have a single account play forever because that would be impossible for a human to do and the most obvious red flag of botting. So creating multiple accounts across multiple sites was the best way to get the most of the bots and cover multiple stakes. I set up different VPNs in the same general area as where the account owner's location claimed to be.

At first I never had any of the bots on the same site and stake play at the same time since I figured it would be sub-optimal to ever have them potentially play against each other. Later I realized that could be a flag and I started semi-randomizing when they would activate and play.


More Automation:
I set up a scheduling system where each bot had its own schedule (e.g. Mon-Thu 11:00AM-7PM with 15% variance). When a bot was set to play, its appropriate VPN was turned on, the virtual machine would boot, the clients would be opened and logged into using a separate script that was able to graphically interact with the virtual machine. Once everything was open a program like TableNinja could be used to register for games that were to pre-defined settings. For sites without TableNinja compatibility I had to write a set of scripts that would check the filters and scroll through to register for the appropriate games.

A forked version of the Table Map system with OpenHoldem was designed to assist with picking games. Going down the games in the lobby or opening up a game lobby the players could be scraped and checked against the database of players and hands to figure out how juicy the game was. I never had this actually stop a bot from joining a game, but I was collecting the data with the plan for better game selection in the future.

Stop losses was a pretty important feature that I built early on. I would have regular iterations of the logic system and sometimes there would be flaws that would make the bot unprofitable. It wasn't always obvious in their play, but over hundreds of thousands of games the losses would prove otherwise. Essentially an account balance check was done at the beginning of play and then every time a new game was registered. If the balance was below the stop loss limit, it would stop registering for new games.


Actual AI:
So the high level logic was something along the lines of:
Evaluate current actions, current stack, potential effects of all available actions, pick the best one.

All knowledge on all the players would be taken into consideration from previous games by utilizing a database of hand histories. In a particular decision the main variables that influenced the decision were current bets by which players, stack sizes of all remaining players to act, the odds of all those players acting in which ways with what kinds of hands, ICM values, position.

For a simple example, if it was folded to on the button it would first evaluate the EV or raising. It first checked the call preflop raise rates of the blinds and overcall rate of the BB. That stat would be determined by any knowledge of those players from those positions at the current blinds. That stat gives a pretty good idea of what kinds of hands those would be, and then it can be calculated with the current hand and ICM equity values what the EV was. Next the EV of limping and folding would be evaluated in a similar fashion and the highest EV score would determine the action.

I build formulas to quickly evaluate all the major stats that would be used for measurement: PFR, CPR, 3Bet, 4Bet, CBet, etc. I would do some statistical analysis to build a dataset that I thought accurately reflected the stat using the variables I wanted. Then I would use symbolic regression with those variables to generate a reasonable formula. This was time consuming and required a lot of computing power. An example formula for cpfr looked like this: f(nplayersdealt,stackbbs,temppos) = (temppos)/(log((temppos)*((temppos)*(((stackbbs) + ((11.601799964904785)*((nplayersdealt)/((stackbbs) + (sin((((-5.0985198020935059)*(temppos)) - (nplayersdealt))/(nplayersdealt))))))) + (((mod((nplayersdealt) + (((nplayersdealt) + ((5.1521501541137695)*(temppos)))/(nplayersdealt)), -1.5626200437545776)) - (mod(nplayersdealt, log(nplayersdealt)))) - (nplayersdealt)))))). It took number of players in the hand, current stack size relative to big blind, and the current position relative to the dealer.

The formulas I was describing above were used to figure out the baseline adjustments of a particular stat based on the most important general knowledge factors. During actual gameplay there was another set of functions that ran in order to weight the stats based on the players' known data. This was grabbed from my shared PostgreSQL database across the bots per poker room. There was also a level of confidence associated with each stat based on how much data I had for it for that particular player.

A major issue with this approach was dealing with heteroscedasticity with the initial analysis. I had way more data on how people play AA than anything else simply because it shows down more often since players are more likely to play it. It becomes increasingly difficult to find out how players play a hand like 22 in rare scenarios, even with millions of hands to analyze.

After awhile of accumulating data I felt that playing against an unknown was possibly the place where I felt most optimized. Not in the sense that I could play better against them than a known player, but relatively to someone else approaching an unknown. Using data I could figure out the distribution of common play tactics of an unknown. One tricky part was you couldn't simply average the distributions by their likelihood of occurring. Playing sub-optimally against one type of strategy could have a far more negative impact than playing sub-optimally against another type. So in some instances it's better to assume the player is going to be aggressive even if it's actually more common for the player to be passive just because it's less harmful against the passive players than it is when they're aggressive.

Improvements were surprisingly slow and undoubtedly where my system needed the most work. After a bot was consistently beating a certain game, moving up in stakes usually didn't go well. It would take quite some time of break even or even negative play before adjustment pushed it into the positive zone. I'd say it took well over half a million hands at each stake to get it where I wanted it to be. Utilizing a great deal of the incoming revenue as a research budget for higher stakes made it so there wasn't very much profit, but kept progress up.


Conclusions:
It was a fun project. My original long term goal was to build a self-improving NL cash game AI and I did spend quite some time building out logic systems for cash games. I figured building SNG bots would be much easier to make profitable, and then once they were running I'd switch over to cash games. I had many of my friends who were profitable at all different stakes and games send me collectively millions of hands to help me out. In exchange I could analyze their respective games and show them very specific instances where they could improve. Having the hole cards of many different profitable players helped immensely with finding core patterns that link to profitability.

To model some of the decision making I put all the players' hand histories into separate databases so that only the knowledge that the player had on the opponents was available. A model I would put together on something as simple as preflop actions from a particular position I could compare with another player's hand histories to see if they would take the same approach using the knowledge they uniquely had on their opponents.

This project helped me learn a lot about programming, statistics, and higher level mathematics. Poker AI specifically isn't something I would ever get into again. For the amount of work that is required there are similar kinds of projects that can yield much higher gains. For example an automated financial trading system. I also can't recommend that anyone else to get into poker botting beyond a hobbyist activity. The number of specialized areas that I needed to branch into in order to piece everything together was much greater than anticipated and often required me to reach out to those more proficient in those areas. Honestly, I think it's something that a strong team should be assembled to pursue and then financially it's likely not worth it compared to other ventures.

0 votes

Comments (21)

Preface:
I wrote this up because some friends were interested in lucid dreaming and I figured I could share what I've learned.

Training to lucid dream is a lot of work, at least it was for me, but I think it's worth it. Dreaming is something everyone does almost daily for their entire lives and it unlocks a vast new aspect to your life.

I got into it more than 10 years ago, but I'd say it took a solid three years before I was able to do so consistently. Even now, I think there's still plenty of room for me to improve. When I first started learning I felt there were very few resources on the subject and I really had to experiment with a lot of strategies on my own to improve.

Step 1: Remembering your dreams
This is the starting point. If you don't remember your dreams, you won't even know if you've had lucid dreams or not. The best way I know of to start remembering your dreams is to write them down when you wake up. This is something extra to do in your daily routine and when you remember more and more details of your dreams it will take longer to do, so there is an immediate commitment involved.

My tip here is when you wake up, don't move a single muscle. Just recall your dreams the best you can and then sit up and write them down. You'll find that your ability to remember dreams more vividly increases quickly and soon you'll be able to remember more of your dreams. If you're sleeping 6-8 hours a night you're probably have 3-4 dreams. There are also non-REM dreams which are more auditory than visual but seem more difficult to remember unless you're waking up from them.

If for whatever reason you just can't seem to remember your dreams, an effective yet unpleasant technique is to be waken up when you're in the middle of REM sleep. You can have someone else do this, wear an EEG that's programmed to trigger your alarm during REM, or just randomly have an alarm go off about an hour after you've fallen asleep and hope you're in REM.

Step 2: Realize you're dreaming
This was the most difficult step for me and I utilized a number of tricks I had read about online to help with it. I think this took at least a year before I had this step working consistently, but over that time period it became more and more frequent.

I've since read that you can use external stimuli when you're in REM as your trigger. For example playing a song or a recording of your voice when the EEG you're wearing knows you've gone into REM. I have never used this, but I can certainly see it working since external stimuli definitely affect your dreams.

I set a number of triggers while I was awake to be used while I was sleeping. I wore a watch and looked at it very frequently throughout the day. Every time I did, I would pause for a moment and think, "Am I dreaming?" When taking a similar action in a dream, I would realize that I am in fact dreaming and would go lucid. I also would reread my dream log on occasion and try to find similar illogical patterns that occurred. Recognizing nonsense would trigger a lucid dream for me as well. There are illogical things that help give away you're in a dream pretty consistently such as looking at text, looking away, and back again with it being different. Same thing with clocks, textures, signs, etc.

Something else that works pretty well for me is if I'm sleep deprived I will immediately go into REM sleep, something called REM rebound. It's pretty easy for me to go right into a lucid dream if I make that my intent while falling asleep.

Step 3: Staying asleep
Every time I realized I was dreaming I would wake up right away. This is pretty awful, especially if it's in the middle of the night. One technique I remember reading about that worked pretty well was to start spinning. Just hold out your arms and start spinning around. The other thing that works for me is just freezing everything. I also have a particular place that I go to often when I first go lucid, but I'm not sure when I started using that. Something else that is pretty effective if you have some control of your environment is to generate a rope that's attached to the ground and just grab onto it. Someone once told me they put their hand in their pocket and grab onto a rope in there that is anchored to the world which is the same concept. Another technique I've read about is to rub your hands together, but I haven't tried it.

My tip on this step is just keep things simple. Don't start trying to build worlds, summon creatures, and fly around quite yet. Anything that was exhilirating and would increase my heartrate substantially increased the difficulty of staying asleep. For example trying to have sex is incredibly difficult to stay asleep through.

I would say even after I became pretty good at staying asleep, the actual lucid dreams don't seem to last very long. On average it feels like I'm lucid for approximately 5 minutes. Eventually it just kind of fades out while you shift out of REM and then when your next dream starts you're back to square one. I'm sure the actual time into the dream that you go lucid has a large impact on the time you have. Something that helps make dreams seem to last for many times longer is letting yourself fall back into the dream. Go lucid, set up your scenario, forget that you're dreaming, and then continue on. Sometime I will come into and out of lucidity a few times in the same dream when I feel the need to change the direction of events or redo something.

Step 4: Experiment
Fly, break physics, be in multiple places at once, try out synesthesia, go talk to yourself (this one is weird), try out some superpowers. Switch out body parts or body entirely. Build a house, city, world, solar system. Rewind time and do something different the second time. Visit places and events from earlier in your life. Make up a completely new life and set of places. Don't exist at all and just watch time unfold.

One of the most difficult barriers to get over with experimentation in a dream is the logic bounds you have on how the universe works. For example, breaking physics was very difficult for me. If I throw a ball or launch myself at high speed, just suddenly stopping mid-air is pretty difficult to do because your brain wants to take physics into account. Anything that would cause you physical pain in reality such as shooting yourself in the foot is extremely difficult to deal with in a dream and takes some effort to get over. If you really want to break the limits of what you can do in your dreams I think you can systematically break down most of these mental constraints.

Early on I was so excited about lucid dreaming that I would go to sleep with a plan on what I wanted to do. One of my early projects was to build and visit a memory castle. A place I could walk around in and place items in different rooms. Awake I have a memory of this place I visited and can envision the different rooms housing different items that are associated with things I want to remember. Later on I more often went to sleep without any plans and discovered that my sleeping self has different and more simplistic desires.

There are a couple examples of things I've done while lucid dreaming that my awake self would consider a waste of a dream. I spent one dream stacking tuna cans higher and higher and needing to fly to continue stacking them. I tried reading a book out loud which was ridiculously difficult. I messed up pronunciations and would stutter, also many of the words I read weren't real. I got half way down a page of a regular paperback-sized book before I woke up.


Just because you're lucid dreaming doesn't necessarily mean you have full control or complete understanding of what's going on. Your subconscious is still going to render the majority of your surrounding environment including people and places. For better or for worse people may still say or do things that surprise you. You may wind up in places or meet someone you weren't expecting to. There's a city I've been building out for many years that I can go to if I ever want to remove myself from a scenario. I think it's a good idea to have a safe and familiar place that you can always use as home base.

0 votes

Comments (8)